EPICS V4 WG Face-to-Face Meeting 2015-05-18

Agenda

  1. Decide on Agenda (today, Friday morning)
  2. V4 Overview on Thursday afternoon
  3. pvaPy and EasyCPP API
  4. Release Planning
  5. Move to git and/or github?
  6. Merging V4 with Base
  7. Subscription Options & pvRequest (related to dbGroup)
  8. Normative Types (preferably with Greg)
  9. Future work (from Greg):
  1. Complete multicast
  2. dbGroup
  3. Gateway: Develop requirements, start on a prototype
  4. Complete IOC interop symmetry (pvAccess link support)

Minutes

Present: AJ, DH, MK, RL, MD, SV, JHL, SH, AA, GG

Scribe: SV

Decide on Agenda (today, Friday morning)

Some items added.

V4 Overview on Thursday afternoon

AJ: What is the status?

MK: Working on developer guide.

DH: We can show outline plan, not ready yet, but will be.

MK: I will be giving a talk.

DH: We’ll have 2 1 hour sessions.

MK/DH: working out between them who will do what….

SH: Are you aiming for people using it in production?

MK: This will be for programmers to start creating v4-based tools.

AJ: No help needed?

MK: Nope.

pvaPy and EasyCPP API

SV: MK has new EZpvACPP API out there, have been using this in pvaPy. Compiles, in testing phase. Plan to finish support for unions, other code changes from Klemen. New API is much easier to use.

MK: Separate directory to let SV see what’s different.

GS: Next version of pvaPy based on this?

SV: Yes.

MK: pvAccess API is very much callback driven. The EZ API takes care of the callbacks. Python user will not see any difference.

SV: Everything will be backwards compatible, internals are more maintainable, can remove classes used to come up with same API as Marty’s new one.

GS: What about stuff not supported?

SV: Nothing will be dropped.

MK: ChannelRPC not supported in EZ API

SV: Will use old method for that support.

GS: If I implement functionality using old API

AJ: Can still use in conjunction with the new API.

GS: Need support for all the NTTypes

SV: On my list.

MK: Look at NormativeTypesCPP

SV: Yes.

Arman Arkilic: We have some users who would like quick reference, simpler document

SV: I’m happy to respond to emails. Need somewhere for the Sphynx documentation

AJ: Add it to the V4 website, talk to MRK about doing that.

MD: Separate python wrapper, I have suggestions about this. We’ll talk offline.

AJ: What about “easy” in module names?

MD: I am in favor of dropping easy, combining this with pvAccessCPP/

MK: Whoever wants to do it, feel free to rename it.

AJ: Does it make sense to have the new functionality in the regular API, e.g  “getDouble()”.

DH: Convenience methods should not go into regular channel API.

AJ: We need to rename “easy” no later than the end of this week.

GS: What about “easy channel RPC”?

DH: This was always part of pvAccessCPP.

RL: In favor of multiple flavors of API.

SH: How many “APIs” will be supported?

GH: Perhaps we should put it into base pvAccessCPP.

SH: If python API is built on top of this, we need to maintain it.

AJ: It will be part of the release, need to come up with a names.

RL: Maybe it’s layer, use numbers? Something a bit more generic that is extensible.

AI: Need to come up with a decision on naming by the end of the week.

Merging V4 with Base

AJ: Managed to build v4 all modules without pvcommon.

RL: boost stuff that is huge.

AJ: had to change pvaccess not to use u-benchmark. Area detector starting to use boost, so I created epics-boost package. idea is that for linux you do not need epics-boost.

MD: Do we want to maintain our fork of boost?

MD: what about vxworks?

AJ: vw still do not use tr1.

AJ: we needed shared pointer stuff.

AJ: bootstrapping is tricky, but not quite that bad.

MD: rtems 4.9 does not have tr1.

AJ: embedded architectures will need boost modules.

AJ: do not want boost in the base

MD: we could have our own shared pointer…

AJ: version of gcc we got we rtems, etc, producing shared pointer would not be too hard.

MD: we got to be careful, and think about atomic ops.

MD: not clear which versions will build for which architectures.

AJ: Strategic path forward: For those archs that do not provide shared pointer we would provide our own implementation of shared pointer. Aim for 3.16?

MK: Do we assume C++ 2011?

AJ: Eventually… We dropped support for vw 5.

Greg appeared on the screen. Cheers in the room...

MD: not sure about our plans for rtems.

MD: VME crates the only places using rtems…

MD/AJ: discussion about board versions and rtems that SV can’t follow…

AJ: issue it talking to existing hardware, need to support it at APS.

MD: VME bus handles endianess for you.

AJ: Moving on…

MD: had complaints about v4 api design. we need to address those issues before merging.

AI on MD: Write latest iteration on complaints regarding v4 APIs.

MD: Will not happen this week.

AJ. Fine, but unless we know about those, we can’t deal with them.

MJ: Completely generic container is hard and not efficient to use. Would be able to deserialize into NDArray without intermediate structure.

MJ: Just example on how to do it would be great.

SH: Deserialization would happen directly off the wire?

MD: Yes.

MD: I am proposing changes to pvData, not to network parts.

MK: Do you want ability to override deserialization methods in pvData?

MD: Key part is to replace factories. They are singletons and can’t be replaced. If we could replace those, and be able to plug in our own factories, changes would not be hard. Would have to be on a per channel basis.

AJ: In other words, PVA API does not go deep enough into the stack…

AJ: Any other things about merge into base?

RL: Any idea how would this fit into base?

AJ: Not completely figured it out, but planning to have it modular so we could mix/match, use only v4, etc.

SH: Would this be after 3.16?

AJ: 3.16 should be by the end of this year, will not include v4 modules. There may not be a Base 3.17, we’d release Base 4.x after the merge.

<Agenda planning with Greg>

GW: Can we do this now - after your break:

  1. Normative Types (preferably with Greg)
  2. Future work (from Greg):
  1. Complete multicast
  2. dbGroup
  3. Gateway: Develop requirements, start on a prototype
  4. Complete IOC interop symmetry (pvAccess link support)

Also interested in “Merging V4 with Base” but more to learn than contribute.

Normative Types (preferably with Greg)

DH: NT doc was updated last summer to match what is in the code. We came up with versioning system. MK updated doc, DH did NTNDArray. Linguistic conventions do not make sense. Hybrid of pvData and BNF do not mesh. Example is time_t. We got duplicates, things in the wrong place, and things that are just wrong.

DH: Looks like pvData metalanguage is easy to use, so use that solve duplicates issues, etc. Use set of linguistic conventions to emulate BNF. Doc has been cleaned up, unions added, conventions clarified.

DH: MK added NTMultiChannel, DH added NTScalarMultiChannel.

DH: I would like this doc to be the dated version.

MK: Second that.

GW: I agree. Lets change links to make this default.

RESOLUTION: Adopt new dated version normativeTypes_20150316.html as the new published version of Normative Types.

All: Agreed.

AI on DH: Change links, etc.

Future work (from Greg):

Complete multicast

GW: MS is almost done with multicast, but “almost” has been there for a while.

RL: Multicast is UDP

GW: What is Bob’s opinion on this?

MD: Do not know…

AI on GW: Find out status from Matej, what his plans are, etc. Include Bob on that.

dbGroup

GW: Many unanswered questions on this topic. Can we make some simplifying assumptions?

RL: My implementation idea is to provide interfaces to do stuff programmatically.

RL: We need pvRequest to have client define how processing dbgroup would take place. This is still not specified.

MH: Do we have to have client define it for the first pass? Why not server define it?

RL: This is one more option… thanks.

AJ: What use cases do we have?

GW: I wrote use cases couple of times. Getting frustrated that they are not taken. E.g., physicist wants to get x/y/current from BPM in one go.

RL: Do not want to put specific implementations down into protocol.

GW: How do we move ahead?

AJ: Where are the use cases?

GW: In emails, epics meeting, etc.

AJ: Do not remember specific.

AI on GW: Find old use cases and publish to the group list.

See thread dbGroup in 4.4 and dependency on IOC EPICS version (May 2014)

https://sourceforge.net/p/epics-pvdata/mailman/epics-pvdata-devel/thread/09C10186-5311-49A4-A9BB-6DA0E1FE9955%40slac.stanford.edu/#msg32387189

Examples;

i) Get the X, Y and current of a BPM with one PV:

 $ eget BPMS:BSY0:23:SUMMARY

     BPMS:BSY0:23:X    0.23 [mm]

     BPMS:BSY0:23:Y    0.18 [mm]

     BPMS:BSY0:23:TMIT 9.45345345E11

ii) Or get both the desired field and readback field of a magnet in one PV:

$ eget QUAD:LI15:56:SUMMARY

     QUAD:LI15:56:BDES 1.5353 [KGm]

     QUAD:LI15:56:BACT 1.6013 [KGm]

[0] http://sourceforge.net/p/epics-pvdata/mailman/message/32385449/

[1] Minutes 1/14/14, https://docs.google.com/document/d/1RJTpjakHsxrcU6-F3UcsV18D8gDhoSdD4GCNRmwFYMA/pub

AJ: If we do server side config only, we simplify things.

AJ: We need put to the group.

MD: What do you need Ralph?

RL: Time/cloning machine.

RL: Hard to get started on something where we have something concrete for specs...

RL: I can write dbgroup without promise of operations being atomic.

AI on RL: Do something.

Gateway: Develop requirements, start on a prototype

AJ: Whos gonna work on this?
GW: Timo there?

GW: With pvDatabase reaching maturity, people will be asking for gateway.

AI on GW: Find Money for gateway.

GW: Has excuses why he has not been pushing…

SH: Requirements?

RL: There is a list that is covering low-level (packet level), and high-level (understands data). Needs to be refined. Something for MS or MK

http://epics-pvdata.sourceforge.net/doc/pvGateway/requirements.html

MD: Anyone who’s got experience with low-level protocols would work.

JHL: How much money is needed?

AJ: Ask MS?

MK: MS has no support for v4 work. PSI has him working on other things.

AJ: Someone from cosylab will be here.

AI on AJ and JHL: Get together, drink beer, and discuss money.

MD: Talking about money without Bob?

Complete IOC interop symmetry (pvAccess link support)

AJ: IOC is not pvaccess slave, can’t do client support.

GW: For example, if feedback IOC could reacquire new matrices, etc.

AJ: You could have external client that could retrieve data.

AJ: We will implement device support that can talk pvacces.

MD: This is gonna get messy.

AI on GW: Find someone who wants to get started on v4 and do this...

MD: This would be prototype for pvalink type.

Use cases:

  1. Feedback IOC needs R12s of feedback after new model run
  2. Magnet IOC wants new polynomials of magnets after calibration db updated
  3. RF IOC wants new field maps after recalculated fields

         

GW leaving before getting more AIs….

--- Break for Lunch ---

Move to git and/or github?

The main issue here is the large amount of HG-specific process descriptions that the group have developed. Converting these to use Git would be a lot of work and is probably not useful for us to have our own versions of these documents, since many tutorials are already available.

The SF issue tracker is crappy and we don’t really use it (probably as a direct result).

MD: Is there anyone here who sees value in having the detailed copy-past level documents that are targeted for developers?

SH: Not for developers.

RL: “How do you do a -pre1 release of V4”

Nobody answered in affirmative

RL: Would we go from our version of flow to the git

MD: Very much in favor of the private branching (github & launchpad model). I have been pushing stuff up to github while I do it, for backup and syncing between computers.

DH: What will we do about the website, stay on SF?

MK: Keep.

MD: Yes, just update repo links to point to github.

RL: SF will also accept forward pointers from other web domains. Not sure if we can make clean links on github. I would move the website repo (pvDataWWW) to git & github, but keep the jenkins job that installs it onto the SF website.

MD: Who’s going to go through the docs deciding what we keep and what we chuck?

DH: Some will have to be updated

RL: User-level doc must be updated

RL: Thinking about git flow. See value of private branches, but would like release & support and main integration branches to stay centralized. Just move the feature branches to private clones.

MD: Agree, that’s what I was thinking.

RL: Keeps intact the whole process of release tagging and branching and support branching that we have already developed.

AJ: How to move forward?

MD: Git conversions happening already, pulling all branches. Look for github repo called “hg2git” for the shell script that does this. Currently run manually.

MK: I want a one-time conversion.

SH: Yes.

RL: Private branch, merge request, we pull this, does this disappear if they delete it?

MD: The history stays in git. The github conversation might go away, but we have all the history in the git repo.

Arman: I have deleted some stuff and it’s still here…

AJ: Other steps?

MD: Migration plan was posted in January, conversion has not corrupted anything. We can switch Jenkins to point to github any time, then its the question of the documentation.

AI on MD: Go through docs and work out what needs changing, deleting.

DH: Release scripts?

AJ: Should be fairly simple changes, one-for-one in most cases.

RL: I have Jenkins job scripts that do many things, in the pvDataWWW repo. Separate project for the Java stuff.

MD: Converting the website would be an excellent project to try out the conversion.

RESOLUTION

We will convert to git & github, Michael will begin the process and manage the conversion process.

DH: Default branch?

MD: Queries remote branch to find what it is

RL: We still use part of git flow thing, should we keep the meaning? <lots of confusing stuff about what we did with branch names>

MD: The tools have files that can configure the branch names.

SH: One-page document describing what we are doing.

AJ: Should try to get closer to the git standard.

RL: Agree.

RL: Distribution of tarfiles?

MD: Stay on SF.

Release Planning

DH: When will we do the next release?

MD: Conversion should wait if there’s going to be one in the next 2 months.

AJ: Do we have a release pending?

RL: Bug fixes for SNS, pvaSrv now builds against 3.15.

AJ: What version would this be? Bug fix?

MD: Bug fix, i.e. 4.4.1

AJ: When?

RL: 4 weeks according to our process (pre1, rc1, final)

SH: We’re using head as of 2015-01-12.

MD: Interaction with Jenkins jobs?

RL: Optional, can do manually.

MD: In favor of getting bug fix releases out quickly.

AI on Andrew: start the release process for 4.4.1

MK: What about new stuff, 4.5?

RL: Plan what’s in it?

MK: easyPvaCPP, pvDatabaseJava, migration to github

RL: The latter is not part of the code, user doesn’t care.

MK: Will develop the two new modules on github, they are already there.

RL: Jenkins jobs?

MK: Not yet.

SH: how long will conversion take?

MD: Depends on availability of AJ and RL for Jenkins job conversions.

AJ: Will create new Jenkins jobs for the git versions. Will start the 4.4.1 process this week if I can.

MD: Will start looking at the conversion steps ASAP. Complication will be changing the git to hg commands.

RL: Try to make as many paths relative as possible.

MD: 4 weeks after starting the bug-fix release process hopefully, maybe 5. Please tell me any HG repositories that I am not already mirroring.

Subscription Options & pvRequest (related to dbGroup)

AJ: was not able to start the AI from the last F2F meeting at SNS, will fit in ASAP.

Any Other Business?

None raised.

Meeting closed about 3:30pm, after internet connection failure.