Minutes of Meetings for vObserver Meetings

The next meeting will be: 2pm AEST Friday 17 June.

Prior to the FunnelWeb team meeting about this, the vObserver architecture was discussed between Chris Tinney, Michael Ireland, and the TAIPAN team, the Michael Ireland and the AAO software team (Nuria Lorente and Keith Shortridge) on 2nd Feb 2016. Then it was discussed again at another TAIPAN meeting on 13 April, 2016 - these notes were placed on the Operational Concept page on the Wiki. The FunnelWeb team, including new half-time AAO employee Duncan Wright, have agreed to lead the vObserver software. The ANU software project to help this process formally started with the kick off telecon on 29 April 2016.

Timeline from April TAIPAN meeting

  • May 12: Scope of work for the vObserver defined through interactions with the TAIPAN team, including input configuration file format.
  • July 12: Final draft architecture document delivered.
  • August 12: Interface to AAO software frozen.
  • September 12: Final draft software framework delivered. This should coincide with including a baseline vObserver, mostly using labour from Duncan Wright.
  • October 12: Testing and iterations of a baseline vObserver complete.

30 April, 2016

Present: Michael Ireland, Jon Nielson, Chris Tinney, Duncan Wright, Sarah Martell

  • A general discussion of the background of the FunnelWeb survey and the vObserver (mostly documented on the Operational Concept page). CT mentioned that it is important to have extra modules, for e.g. quicklook data reduction and planning, be separate from vObserver. MI noted that this was the architecture outlined in the last TAIPAN meeting, with Ned Taylor calling these "loops".
  • A brief discussion of different kinds of architectures that JN will have to consider, e.g. a formal Finite-State Machine library or nested if statements that achieves the same purpose. One process or multiple concurrent processes for the scheduler (e.g. on slew, would vObserver wait for everything before going to the next step, or would it go to the next step independently on different subsystems). JN pointed out that important considerations would be the detail of the observing logic, and keeping the code easy for scientists to modify with a very low risk of breaking the code.
  • FITS header information may be complex to obtain. CT stated (and all agreed) that scientists need the ability to add to the fits header at a later date, which means that a pure DRAMA task may not be ideal. No-one objected to the archiving being done by a separate process controlled by vObserver.
  • Given that vObserver has to not only communicate with the AAO software, but at least 2 or 3 pieces of collaboration software, it will be important to define our own communication standard. Although it would be ideal for this to be the same standard as used by the AAO, this restriction is not necessary and we can work in advance of them.

Action Items:
  1. JN to speak with Nuria Lorente about the AAO/vObserver interface in more detail.
  2. JN to initiate a document structure (here on Wikispaces?) to define and iterate with scientists on a baseline observing logic.
  3. ALL: (especially DW/CT) add to this structure.
  4. JN to begin an architecture document, in local consultation with MI.

13 May, 2016

Present: Andrew Hopkins, Mike Ireland, Sarah Martell, Jon Nielsen, Marc White, Duncan Wright

Previous Action Items:
  1. JN has emailed Nuria. Andrew has spoken with her and a meeting has been arranged for Tuesday at 2pm [though 1pm would suit JN better].
  2. The baseline observing logic [now renamed the vObserver Timing Diagram] wasn't yet started, but MI created the page during the meeting and will fill in some details in the coming week.
  3. Others will add to the timing diagram as appropriate.
  4. A first draft of the vObserver Architecture is done.

There was some discussion of timescales for the project. MI put the timeline at the top of this page. AH said that science verification is intended to happen in late 2016 or early 2017 and so the software needs to be ready well before then.

The Observation DB and the relative merits of SQL were discussed. Since the Nightly Planner will be using SQL (MW and Ned are working on this) it makes sense for some aspects of the vObserver to also use it, and it's useful to be able to keep track of observations as they flow through the system. However the data model for the database(s) is not trivial. This will require additional discussion.

The baseline vObserver Scheduler does not have to use SQL but will instead work its way through a list of observations, with only minimal checks on observing conditions. This list of observations could just be a text file output from the Nightly Planner. At a later date the intelligence in the Scheduler can be improved if required. This could involve having separate FunnelWeb/TAIPAN observing queues so that time could be allocated more flexibly based on observing conditions.

The definition of an "observation", which goes a long way to describing the interface between the Scheduler and the vObserver, was discussed. We need it to specify everything that is required for an observation. Also, we noted that an observation will consist of some number (at this stage 3 x 5 minutes) of exposures, plus associated calibration frames. It's not required to take calibrations with every observation, but it should be possible. The concept of keeping track of "stale calibrations" was mentioned. Something in the system needs to be able to keep track of this.

The Archive loop was discussed, and the strategy of being able to add extra FITS headers to the files that are written from the AAO software was seen as a sensible compromise.

The QA loop was discussed - the initial design seems ok at the very top-level, but there will be many more specifics to be determined. There will be differences in the QA between the two surveys. At this stage it doesn't seem like the vObserver group need to worry too much about this.

Action Items:
  1. JN to remotely attend meeting with Keith, Nuria, Tony, and Andrew re the DRAMA / Python interface on Tuesday the 17th
  2. MI to complete his first draft of the vObserver Timing Diagram
  3. ALL to review that diagram
  4. JN/MW to initiate exploration of python interfaces for tables in the ObservationDB for the nightly observation list (for the Scheduler) and the field observation status update (for the Archiver).

Next Meeting: Friday 27th May at 1pm. DW indicated he may not be available that day.

27 May, 2016

Relatively brief meeting with Jon Neilson, Mike Ireland and Andrew Hopkins. Mike also met with Tony Farrell and Keith Shortridge.

31 May, 2016

Present: ???

From Andrew:

No formal notes from today's discussion, but just quickly:
- We introduced Carlos Bacigalupo, the new AAO software person, who will take on the task of developing the python interface to the DRAMA layer, and in the first instance defining the APIs.
- Nuria provided updates on developments within the AAO software team, in particular that Tony is looking into the issue of threading with python, and will provide an overview of options shortly.

Actions for now are mostly on the AAO software team side, and they are well on top of what needs to happen.