Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.eso.org/projects/dfs/team/JP2PP-test-report-V2-2int2.txt Дата изменения: Wed Oct 24 19:36:09 2001 Дата индексирования: Sun Apr 13 22:54:26 2008 Кодировка:

TEST REPORT FOR P2PP V2_2int2
=============================



Tests performed from Oct 10 to Oct 18, 2001.

Running on HP-UX 11, jre 1.1.8 with VLTMARCH2001 (SEG test machine called 'wu0oh'),
accessing SEGSRV12 in direct mode, account 'visitor', user '1901'.


Performed tests
---------------
Main goal was to check:
(a) some rather extensive infrastructure changes, which result
in menu items being enabled and disabled as needed [0884];
(b) fixes for all known problems related to the 'paramfile'
parameters.

Other main functionalities of P2PP were also verified, but not as deeply as with
the June/July beta-releases:
* Manipulation of OBs (create, duplicate, move, delete, verify), for science and
calibration OBs mostly using FORS2 templates, and some with ISAAC.
* Import/export OBs.
* Save as OBD format.
* Synchronize OB components.
* View phase 1 targets and download target definitions.
* Check in / check out, directly accessing DB.
* Generation of reports (execution time, ObsBlock breakdown).
* Interface to BOB (fetch of OBs, load OBD file generated by P2PP).
* execution of the Verify function, but without EVM scripts.

I didn't try to retrieve caches created with previous releases (V2_1_5 or V2_2betas),
or to see how the tool behaves when TSF and IMPEX files are "somehow" wrong
since this situation was rather extensively tested in June/July.
Super-user (user '0') has not been used.


Conclusion
----------
This test session represents an effort of about 18 hours spread over 7 working days.

On total, 11 new bugs were found:
* 6 High
* 5 Minor

One important issue is about inconsistencies between a target paramfile and the
TargetPackage. Indeed V2_2int2 solves several problems previously detected with
beta10-11-12, but I think it's useful to mention how easy it can be to get
such inconsistencies, for instance when:
- attaching by mistake a second p_targ file (when a p_focf or a p_gbr is expected)
- downloading a target definition from the "View Phase1 targets" menu item
- or simply when modifying by hand RA or DEC in the TP
And even if EVM or the user himself detects inconsistencies, there will be no
way for him to automatically retrieve the correct values from the paramfile: he must
manually re-enter values in the TP.
So, I would suggest to clearly explain this possibly surprising behavior to Paranal
before delivering V2_2 (in the official V2_2 Release Note, or even directly
discussing with ScienceOps ?), then get some feedback from them using this feature,
and finally re-discuss this issue in a next P2PP SCCB meeting.

Other main bugs to be noticed (which should be fixed for the next beta early
November) are:
- Synchronizing an AcquisitionTemplate which attaches a p_targ file doesn't automatically
update target values in TP, as it should.
- Synchronizing AT, OD, CS and TP doesn't preserve the "execution time" value, as it
should.
- the "Verification error log" box displayed when running Verify doesn't give the OB name
and the count of correct OBs as it did before (this is a regression).
- Synchronizing an OD to a Calibration OB makes P2PP to hang up.



Detailed test results
---------------------
Comments from Karim, replies from Maurizio.

1)
MINOR:
installation instructions on the web
(ftp://ftp.eso.org/pub/users/uss/ohs/jp2pp/INSTALL-2.2int2.html)
should ask to run "jre -version" instead of "jre -help" to see
the current jre version (jre -help just returns possible arguments of
the jre command.
> Unfortunately the "-version" command line option is not available on
> SunOS and Linux: the JRE version is output by the "-help" option.
> Note also that the "-version" option of the HP-UX JRE gives the *build*
> version (something like "C.01.18.03"), not the Java language version....
==> I'll try to re-word the instructions anyway.


2)
about the "Folder" menu: if P2PP is running for the first
time (local cache is new and thus empty), top folder is selected, here
following "Folder" menu items should be in grey since no
specific folder is selected: "Rename", "Move" and "Delete" (only keep
"New" and "Open" here).
>This is difficult to implement, and in any case the commands you propose
>to gray out do put out an error panel if the get invoked.
==> no action


3)
MINOR:
why put the "Reports" menu itself in grey, while the
"Synchronise" menu is not (but all related items are in grey) ? I would
suggest to keep a menu header in black even if all its internal menu items are
to be grey.
==> Item 1537 for next SCCB meeting.


4)
attach a paramfile to a keyword which is not a paramfile type:
==> OK, the TP values are *not* updated.


5)
create a new FORS2 OB attaching a mxu AT + 1 science template; attach
a p_targ file:
==> OK, the TP is updated according to the p_targ values.


6)
same as 5) but with p_targ file *not* containing any double quotation
for keyword values:
==> OK, the TP is updated according to the p_targ values.


7)
same as 6), then modify by hand via the OB View the RA value. Quit,
then re-start P2PP:
==> OK, the new RA value is correctly stored in the local cache.


8)
same as 7), Verify the modified FORS2 OB: no error reported
in the "Verification error log" box, but following error message in the debug log:
Start!@@@@@@@@@@@@@@@@@@@@@@ We are here !!!!!!!!!!!!!!!!!!!
!@@@@@@@@@@@@@@@@@@@@@@ We are here !!!!!!!!!!!!!!!!!!!
!!!!!!!!!!! Index 3
'import exceptions' failed; using string-based exceptions
I don't think this is important but, please, confirm.
> It is not important, confirmed.
> See also 1486/P2PP.
==> no action


9)
duplicate the previous OB (with modified RA value), and check whether
the new one also gets the modified RA:
==> OK, this is fine.


10) export same FORS2 OB as 9) (i.e. OB attaching a paramfile).
Expected result should be: target values appear in the IMPEX file:
==. OK, it does.


11) create another new FORS2 OB attaching a mxu AT. Attach a p_focf file
where a p_targ one was expected.
Expected result should be: target values are *not* updated.
==> OK, they are not updated.


12) same as 11) but attach a p_gbr where a p_targ one was expected.
Expected result should be: target values are *not* updated:
==> OK, they are not updated.


13)
HIGH:
same OB as 11), but now I would like to simulate following possible error
which can be done by a user:
Let's imagine he attaches a first (correct) p_targ file where indeed a
p_targ is expected, for instance to the FORS2_mxu_acq template.
Then, by mistake - why not ? this can be possible ...-, he attaches
to a second different parameter another different and correct p_targ
file where a *p_focf* one is expected. Maybe I'm wrong but it seems to
me to be a quite easy mistake to be done: select a wrong file in a
directory possibly containing tens of files.

If later on, he runs the EVM, he should get an error message saying
something like "not allowed to attach 2 p_targ files". Fine, So our user
should replace the second p_targ file by a p_focf one. But, as P2PP is
currently implemented target values in TP *remain* with values extracted
from the *second* p_targ, i.e. there are inconsistencies between TP values and
the remaining (first) p_targ file.
But OK, why not since EVM should also detect such inconsistency. The
problem here is that even if the user is aware of such inconsistency,
how in practice can he re-set correct values into the TP ?!

We talked during one of our recent SCCB meetings of a possible new
button called like "re-load target values" to re-scan p_targ, but this
suggestion was rejected if I remember correctly (?).

Conclusion: from my point of view, here is a quite easy sequence to get
an inconsistency between TP values and a p_targ file. Even if EVM
detects it, I think there should be somehow a "mechanism" to
automatically update the TP (we can not ask the user to do it by hand
...).
This mechanism can be:
- the (non-famous !) "re-load target values" button
- a new feature to automatically re-scan all attached param files each
time anyone of the param file is re-attached (easy to implement ?). This
should not impact performances ?
==> Item 1538 for next SCCB meeting. For this release, we'll need to
explain the situation in the known-problems document.


14)
attach a p_targ file to a science template parameter expecting a
paramfile (a p_focf one). Expected result should be: previous target
values are retrieved from a p_targ attached to the AT are *not* modified, target
values only comes from p_targ paramefile attached to an Acquisition Template (not to
a science template):
==> OK previous values are unchanged.


15)
same as 14) but with a calibration template instead of a science
template:
==> OK same (correct) behavior.


16)
create a new FORS2 OB with FORS2_mxu_acq as AT, attach a wrong
p_targ file to it (PAF.NAME keyword value is empty:
==> OK a meaningful error message is now displayed, the paramfile is
not attached, and the OB remain as it was, which is correct (was item 1456
'unclear error message' with the 2.2beta11 July release).


Now running some synchronise about target feature.

17)
already have a first FORS2 OB with FORS2_mxu_acq as AT, attach a
correct p_targ, OK the target values appear in TP. Create a new FORS2
and copy/paste TP from first to second OB:
==> OK values are correctly pasted.


18)
HIGH:
same first FORS2 OB as 17) attaching a correct p_targ file.
Create another FORS2 OB, and copy/paste the AT: OK, the AT is correctly
pasted to the new OB, but target values are *not* retrieved in the corresponding TP.
Here is another trivial example of inconsistency betweeen AT and TP.
Same problem as 13): even if EVM detects such inconsistencies, how the user
can retrieve paramfile values into the TP (by hand ? not so good ...)
==> Fixed, see next release.


19)
HIGH:
create a FORS2 OB with FORS_mxu_acq AT, several science template, TP correctly
retrieved from the AT p_targ, CS entered. Create another new FORS2 OB,
Synchronise AT, OD, CS and TP. Export both OBs and compare IMPEX:
< ExecutionTime "00:13:30"
---
> ExecutionTime "00:00:00"
The Execution Time is not recomputed and displayed by the OB View for
the pasted OB where it should, isn't it ? Does EVM check this type of error ?
Even if it does, how the user can recompute such parameter ?
Notice that Duplicate keeps the original Execution Time value.
==> Item 1539 for the next release.


20)
MINOR:
value for "Execution Time" are now computed from the "Exposure
Time" parameter value and displayed in the OB View, which is fine.
I created a CalibOB attaching a science template called
"FORS2_mxu_obs_slit" which also contains an "Exposure Time" parameter:
no "execution Time" value is displayed by the OB grid: is it normal ?
==> Fixed, see next release.


21)
more about 20) for normal OBs, I don't now how Execution Time is
computed from the Exposure time value, but they behaves in the same way
(if Exposure Time is increased, Exec Time is increased too, and vice-versa ...).
Anyhow, one comment: the "FORS2_mxu_acq" template also contains 2 keyword about
Exp. Time ("Exp. Time for Field Image", "Exp. Time for Slit Image") but they don't
seem to be considered when calculating the Execution Time: seems normal
for me, but please, can you confirm ?
==> Confirmed.


About the import/export

22)
create a ISAAC OB 1 At+1 science, enter values for every TP, CS, OD
parameters. Verify it (it's correct). Export it. Rename the IMPEX with
the "orig" prefix. Import it and re-export it. Compare the 2 IMPEX:
==> OK, no difference except the name, which is fine.


About check-in

23)
check-in the previous ISAAC OB:
==> error message "Invalid column name
'execution_time'" which is normal since I didn't run the "alter"
statement on SEGSRV12 yet. The OB is fine too (status, ID remain as before).


24)
MINOR:
with previous P2PP releases CTRL-K was to run a check-in. Now
it performs a "Display Object Details". Why this change ?
I find the short cut for check-in very useful for me, and I never need
to use a short cut for displaying the OB (since there is already the View button) ...
==> this is a regression. Ctrl-K will be restored to do check-in, see next release.


25)
Now, perform the "alter" on SEGSRV12.
FORS2 OB with 1 mxu AT + 3 science templates.
Verify from main P2PP box gives no error: OK
Export it: OK
Check it in: OK I get an ID (108358, user is 1901 on SEGSRV12).
Run the DB browser and retrieve this OB: OK I can see it and also the
new 'Execution Time' column.
Check-out the OB: OK no problem.
Export it again, and compare with IMPEX as generated before the
check-in: NOK
I have a difference (new line in the second IMPEX):
> absolute_times_list "{1970-01-01T00:00:00 2038-01-19T03:14:07 5} "
This bug was already reported by me in July on beta11: was item 1461,
declared as MINOR and originally planned for V2.2.1. ...
==> Item 1461 was closed with no action (Sept/Oct SCCB meetings). This is really
minor and has no impact.


26)
Update by hand target value RA of previous FORS2 OB, there is thus
an inconsistency between TP and the p_targ which is attached. Verify
reports no error (OK, I don't have corresponding EVM). Check in the OB:
no error reported too. OK one can argue it's already the case with 2.1.
I think at least such problem should be described in the V2.2 list of
known problems, isn't it ?
==> see answer for pt 13)


27)
another (quite easy) way to create an inconsistency between p_targ
values and TP, is to run download a target definition from the "View
Phase1 targets" menu item
==> see answer for pt 13)


28)
Execution Time value is not in the "ObsBlocks breakdown" reports: is
it normal ?
> Yes.
==> no action


29)
MINOR:
FORS2 OB called "my first V2-2int2 OB" (user 1901, on wu0oh visitor,
SEGSRV12, obs run: 166.A-0701(C)/SM/FORS2).
View Phase1 targets>Query then Download the target called "C1252.9-2827"
I get following Java Exception, but the target is downloaded, and Verify
says no error. Check-in the OB: no problem (ID is 108359).
READ: id=660701000106(6607010001); type=class org.eso.ohs.dfs.Target
Key = or
CLASS DbaseHandlerTG1 - read: sql = SELECT * FROM targets WHERE id =1
AND programme_id =6607010
CLASS SimpleStorableObject - setId: newValue = 6607010001
Exception occurred during event dispatching:
java.lang.ClassCastException: java.lang.String
at org.eso.ohs.p2pp.dbb.P2PPView.getClassForId(P2PPView.java:94)
...
==> Fixed, see next release.


30)
download same target as 29) on another FORS2 OB (a similar OB but not
exactly the same): no exception reported
==> no action


31)
export a mxu FORS2 OB attaching a paramfile (and with consistency
between this paramfile and TP). Edit the IMPEX filem and modify
the RA and DEC value in the TP level.
Import the modified IMPEX: OK, the paramfile is parsed and the OB View
gives the original values from the paramfile (not the modified values).
Corresponds to the expected implementation as described at:
http://www.eso.org/~uss/ohs/docs/specs/paramfiles.html
(see paragraph '5.Notes', 4th item)


32)
more about my previous 18) comment: I think it's a real bug
according to the
expected implementation of V2.2int2 as described at:
http://www.eso.org/~uss/ohs/docs/specs/paramfiles.html
(see paragraph '5.Notes', 5th item)


33)
in fact I can not test 7th item of paragraph '5.Notes' at:
http://www.eso.org/~uss/ohs/docs/specs/paramfiles.html
("Attaching multiple paramfiles will result in setting Target fields
multiple
times (the last update overrides the previous updates). ")
==> On the latest 15 July FORS2 IP version, only acquisition templates
get the
INS.TARG.SETUP.TYPE "paramfile"; # Keyword type
definition, no science or calibration template with such definition.
==> this is fine in fact: no action.


34) HIGH:
Introduce by hand following error in a paramfile:
TEL.TARG.DELTA 007000
Attach this wrong paramfile to an OB already attaching a (correct)
paramfile: an error message is displayed ("error while attaching paramfile xxx
Illegal DDMMSS value: 0:70:0.0 Target RA and/or Dec was not updated").
and
Following Java axecption:
java.lang.IllegalArgumentException: Illegal DDMMSS value: 0:70:0.0
at
org.eso.ohs.utilities.Convert.checkBoundsDDMMSS(Convert.java:518)
at
org.eso.ohs.utilities.Convert.stringToMilliarcsecNoColons(Convert.java:288)
at
org.eso.ohs.utilities.Convert.stringToMilliarcsec(Convert.java:141)
at
org.eso.ohs.utilities.Convert.DDMMSSToMilliarcsec(Convert.java:112)
at org.eso.ohs.dfs.Target.updateFromParamfile(Target.java:735)
at
org.eso.ohs.p2pp.views.EditorFactory.fileEditor(EditorFactory.java:638)
at
org.eso.ohs.p2pp.views.EditorFactory.paramfileEditor(EditorFactory.java:464)
at
org.eso.ohs.p2pp.views.EditorFactory.editorComponent(EditorFactory.java:376)
at
org.eso.ohs.p2pp.views.EditorFactory.getTableCellEditorComponent(EditorFactory.java:347)

This corresponds to the implementation described at:
http://www.eso.org/~uss/ohs/docs/specs/paramfiles.html (the wrong file
is attached, but previous (correct) target values are preserved).

Anyhow, I would prefer another implementation: if the paramfile to be
attached contains such error, why not simply *not* attach it ? this should be
quite easy to fix since you already catches the error and consistent with our needs: avoid
as much as possible inconsistencies between paramfiles and target values, isn't it ?
Furthermore, if target definitions are to be retrieved from the
paramfile and not from TP at execution time, we should really avoid to attach
wrong paramfiles.
==> Change Request: item 1540 for next SCCB meeting.


35)36)37)38)
HIGH:
Selecting the "wrong OB" from previous 34) item, and finally click on the Verify
button from the main P2PP box (i.e. not selecting the Verify item from the DBB
menu): the Verification error log box reports "0 OB(s) verified without error",
which is obviously a strange message.

Verify always reports same message ("0 OB(s) verified without error") even if applied
on *several* OBs.
Furthermore, when verifying a set of (wrong) OBs the OB name is not displayed as
before.
==> this is a regression: item 1541 for next release.


39) HIGH:
I run thus P2PP once more with following sequence:
- create first a normal OB attaching 1 AT + 1 science template and 2 calib templates.
This is the OB named "big-one-for-OT2.2int2-test", i.e. last FORS2 obs run in my
current cache. No problem.
- Verify: OK no error, no red button.

Then, I wanted to create a "quite big" calibration OB by synchronizing with the first OB:
- select the first OB
- Synchronize>Copy OD
- select the target Calib OB (same Prog ID, this is the first of the two ones under my
current cache, created some days ago)
- Synchronize>paste: P2PP hangs (watch displayed indefinitely), the Paste is never
performed, and Java exception originated:
java.lang.ClassCastException: org.eso.ohs.dfs.CalibrationBlock
...
I can not do anything than File>quit (this works), all other items such as OBView did not
work anymore.
==> Fixed, see next release.