DAY 1, Tuesday July 15, 2014: Version 4.4 preparedness status reports

9:00 AM Convene. Start at 9:20. Precise location to be announced. TIMES ARE AT CHAIR'S DISCRETION.

Status reports and demonstrations of work for planned version 4.4 features.


Attendees: BD, MK, MS, DH, GS, KK, EM, KS, AJ, GC, …

Scribe: AJ

BD: Many people here are interested in services and physics apps rather than just core, and I want to ensure that we cover those in the time they are here. The core group are here on Thursday, so some of those topics can be delayed until Thursday morning. Join the DISCS collaboration with this group for GS’s talk.

GS: MASAR and its Future slides

MS: Timo @ PSI had Cosylab develop a snapshot service, this is different.

GS: PYMASAR talks to SQLite and provides service.

BD: Managing save-sets is a service, independent of GUIs or other clients.

GS: MASAR has always been behind the core V4 release versions, 4.4 is currently being developed but only recently updated MASAR to 4.3.0.

MS: ChannelRPC helper classes were not well tested.

KK: Is there a Python binding to pvAccess?

GS: Wasn’t originally, now using Sinisa’s binding but it’s behind.

MK: Never made Python bindings a priority. Sinisa’s approach is the right way to go.

MS: pvAccess Channel RPC API in 3.0.3 was too simple for GS, needed to update it for his purposes.

BD: Do we have any other RPC service testing the interface?

GS: I do have one (Metadata store) but not in use yet. Using client side of Sinisa’s code for MASAR, but for metatdata store will need both.

RL: There is a V4 service for ChannelFinder, but nobody uses it & it’s read-only (auth*n not implemented in V4).

MS: When you query ChannelFinder you can get 10MB of data, need chunking to limit data volume; iterators etc.

RL: Also need ability to interrupt response.

MS: Need support in core for this. Service-specific though.

KS: Pagination too, for olog. Very long queries tend to be slow, even at 150MB results.

GS: Making extensive use of NTTable & NTNameValue. Now need the post-4.3.0 changes to ChanneRPC API, drop need for pvIOCCPP. Integrated pvaPy and can now use generic eget tool to query MASAR.

MS: pvget & pvput are simple like caget/caput, eget also supports RPC.

BD: Underlying V4 code refactoring has simplified the application code?

GS: Yes, but painful because of backwards compatibility issues.

MS: What’s missing is an NT helper library.

KK:Greg was against there being a helper library for creating NT Types?

BD: Yes, but I’m a pragmatist, we need them.

MK: There hasn’t been a firm definition, what GS is using is not what is in the document for NTTable. I have comments in a later talk.

BD: Need to mimic the VTypes. MK to lead this effort, GC to help, with MS and GS.

MS: GC wants a separate conversion library.

MK: NTTable,NTMultiChannel NTNDArray,NTNameValue, NTScalar.

GC: GS should be able to use CSS to access MASAR, end-to-end from VTypes.

DH: Add me to the group for NTNDArray definition.

BD: I want a group of standard services; streaming data, get & put. GS doing RPC. How do we move these to real services. Trying to get away from huge clients (SDDS, MMT, GDA etc.)

GS: Strong desire to integrate with CSS, but don’t know how. PyQt is a lot of work. Python API for NTs. Operators asking for Python now.

BD: We have to give physicists APIs, they couldn’t wait for our tools, but they do need to converge now.

GS: Want to update to V4.4 release once solid. Problems if APIs are not backwards-compatible, took me 6 months to update, maybe longer.

KK: Still lots of changes occurring, at this point there’s no time to maintain backwards-compatible API. Thing we need is when the protocol is frozen, then maybe the API can freeze after that.

MS: After V4.4 the protocol will not change much, will keep communications compatible between different versions of client & server. Promised, committed to.

GC: API changes possible as long as they get hidden from code using the old API.

- 10 minute break -

areaDetector overview by David Hickin

DH working in transporting NDData over network. Usuaally Windows to Linux.

Several reasons.

Goal  90 I0MB images per second.

areaDector overview. Plugins, High Performance, etc. Base class is NDArray.

DH developed plugin which is a pvAccess server.

Existing solution uses pvAccess network protocol to push data to client but does not use pva API.

New is using pva API, pvDatabaseCPP, etc.

adds compression

new NTNDArraay to replace old version of NTImage.

Future Plans 1) Plugins (now), 2) imbed in AD, 3) Pure V4 areadDetector 4) V4 asyn/V4 database 5) enhance areaDetector. Not sure what should be done.

Can pick of parts to do first without redoing all of areaDetector.

DH gave demo.

End of morning. Afternoon session has its own Hangout.