Telecon Minutes, June 4th, 2013

=======================

See minutes below

Agenda as Followed

0. Preliminaries (5 mins)

1. Convert discussion, if needed
  There has been a lot of discussion on the list. I can't see specific things
  that seem like they're not being handled there, but if there are let's use this time.

2. Data n-type (Bob, 30 mins)
  First proposal for general dimensional data type

3. 4.3 beta planning [2] (20 mins).  -> Was not covered.
   Confirm what will be in 4.3:
     easyPVA?
     exampleCPP and exampleJava?
     1 tar or many as in the past?
     Should anything be removed from pvIOCCPP to make it so it doesn't contain dead wood?
  Web site and doc changes for 4.3 beta.
  Packaging 4.3 beta. Who and when.

[1]
 http://epics-pvdata.sourceforge.net/home.html#usefulinfo
[2] beta 4.3 bundle, and Marty's breakdown of how to integrate pvDataCPP-md
   the release,
 http://sourceforge.net/mailarchive/forum.php?thread_name=407261CB-B96D-4C57-85D0-0DEE68CEF1FD%40slac.stanford.edu&forum_name=epics-pvdata-devel

Minutes

Present: DH, GW, MD, GS, MK, MS, NM, SV, TK, RL

Scribe: MD

Chair: GW

********

1. NEW TOPIC: Q&A on Convert proposal

********

MK: DH, RL, AJ please look closely at the proposal of Michael’s.

MK: Copy on write bothers me for big arrays

MD: Only nec if you have >1 consumer. No need with 1 : 1 prod:cons

MK: MD’s prop is that whenever you do a rite you do a copy:

MD: No, only when there is a need to do a write. When no refs outsstanding there is no need to copy

MK: But when there is an outstanding reference there will be a copy

MD: Well, that’s write, but: When you do a monitor on an array, each thread will attempt to doa copy. I intend to do these copies simultaneousl- which will speed up copy and increase scalability

MK: Not sure how that would be implemented

GS: You must have a lock before reading

MD: That’s right; so one can use read-write lock, or copy-on-write. Either would be ok. To avoid dead-lock, I would advocate copy on write. You only need copy on reference, and that’s handled by the shared pointer.

MK: I feel uneasy about it.

MD: What evidence would you like MK?

MK: I’d like to hear from DH and MS.

MS: If the arrays are large, ???

MK: A target app is v. large arrays eg for image

MD: Well, if you don’t have enough memory, then no approach that requries all in memory would work.

MK: I don’t understand the seq that indictaes not having to copy.

MD: If the cons if faster than the producer,then there will be little need to copy. If the opposite, then there must be copies

MK: If cons are monitors, then one can notify that there is an overrun.

MD: True. And that’s ok

BD: MK is saying if the q size is 1, then …

<discussion>

BD: Can the q-size ever be 0?

MD: Yes, if the …

...

BD: Which behaviour do you want?

MK: The producer must be able to produce things as fast as he wants.

MD: When prod rate is faster than consumer rate, then there won’t be too many copies.

MK: … it’s ok for the producer to simply notify consumers that it has not kept up.

BD: The ability to optimise for throughput or to optimise for retaining data, both exist in MD’s scenario.

You would optimise for retaining by queueing (but adding latency on the side of the producer), whereas

you would optimise for throughput by not accepting data (eg not accepting new frames until the existing is consumed).

MK: I see problems in the details.

MD: There is no reason you would have to use ViewUnsafe for a copy; you can use reuse(). This checks data is exclusively viewed by the caller, and so copy can be done safely.

MK: .. There will be cases when what is referenced is not unique.

MD: But we make it (the data) unique by making a copy, when you must write to it.

MK: Example: code sitting under a record, takes d from a camera and puts into an array. Your proposal implies a copy.

MD: No, only potentially makes a copy. Only if there is another outstanding reference.

BD: Only make a cop

MK: What’s the programmatic decision about allocate more mem?

MD: Take out reference, then inspect with method “unique()”, to know if multiple people sharing data. If there are other people you MUST allocate more memory, if there are not, you would not allocate more memory.

MK: A problem is the code getting data from camer, has no idea how many clients there are

MD: But you do want the client to decide on flow control

BD: I think MD’s prop for flow control is what should be used in area detect. This would stop AD doing invisible things underneath

<discussion on flow control normally in image processing, to establish suitability of MD’s proposal>

MK: I’m not sure you want to be doing processing of decision making on splitting up the processing to multiple threads at record level!

BD: I explicitly do think we should be doing that at record level.

TK: Q: In scenario where Eiger detector gives a lot of data, which is being saved. Then a poor client that can’t keep up with consuming, would that slow does the archiving (other client).

MD: If the producer is poorly written too, then the poorly written client may slow down the archiver, yes.

BD: [The framework supports well written systems greatly, at the expense of not supporting badly written systems. That’s ok].

BD: We should have stated the requirement, and checked MD’s proposal against the requirement.

BD: I see AD as the requirements that the EPICS Db did not meet. Same for GDA.

*********

NEW TOPIC:   Data n-type (bob)

**********

Pertains to a General Data Normative Type.

BD: <Makes presentation>.