Agenda

1. pvRequest review [2], *Marty*, 40 mins.

  Everyone please read [2] and be ready with analysis.

 
*Ralph* please check [2] gives you what you need for pvaSrv.

  Desired outcome: what needs to be implemented to complete the support
  of specification of a request, esp w.r.t. monitoring. Where will it be
  implemented (given that pvIOC is no longer the primary server side) - is
  the pvIOC data model independent of use of a pvIOC in the server? Who
  will implement it?

2. NTImage and the standardization of images communications over pvaccess.

  + First reviews.
   -Review of David's revised NTImage (10 mins) [3]
   - Review of Nikolay's proposal (20 mins) [4].
  + Nikolay, can you describe how your NTImage would be used to encode
  an AreaDetector image [5]. Have you done this already?
  + Whether NTIMage should encode an image, or wrap an abstract encoding?

  Desired outcome: Ensure that in the short term, eget can decode and display
  images produced by the test image server of areaDetctor sourced images. Decide
  whether, in the longer term, we intend to standardize a one structure definition
  for images ("NTImage"), more than one (that is, one specifically for
  areaDetector, and another more generic one), or even, one wrapper
  of an arbitrary number of encodings (like TIFF vs JPEG).

Minutes

Present: AJ, DH, GW, MS, NM, RL

Scribe: GW

Chair: GW

pvRequest review

deferred as neither Bob nor Marty are present

NTImage and the standardization of image communications over pvAccess

Review of David's revised NTImage [3]

DH: Didn’t like hard coded scalar array. But it’s not configurable, and in AD apps the definition can change. So, hard coded type is not good. NTVariantArray might work, but DH proposes the

simpler byte array and another variable which gives the type in the byte array.

The two fields “ColorMode” and “BayerPattern” - not clear whether this replaces the same named fields in NDarray, or different. If one views the NTImage as generally an image, then

they may be necessary, but if NTImage is more for NDarray then they’re not needed.

MS: When looking at AreaDetector, I saw statements like switch (ColorMode), but MS found he didn’t need to use BayerPattern. So, MS suggests ColorMode is pushed up, and BayerPattern is dropped from NTImage.

DH: If ColorMode is made a field in its own right, it should still remain in the attributes list. DH - preference is it should remain an attribute.

DH: Preference would be that NDarray should be plainly streamed on the bus. That is, lossless transport of an NDArray. Make its encoding into an NTAreaDetectorImage and out again as simple as possible.

[DH: AreaDetectorImage wraps an NDArray]

MS: +1, keep ColorMode in attributes. If ColorMode is not in attributes, then a general purpose client could not decode it.

DH: If it’s not there, could you not just assume 0 - greyscale.

MS: Yes, ok.

MS: To sum up, if there is no ColorMode we will assume greyscale.

DH: Also suggest dropping fourcc [because we don’t know what it means]

DH: Things size_t in NDarray I made uint, but they can be ints (to support java).

Revised proposed structure, as before, but with with uints changed to ints

structure NTAreaDetectorImage  // was NTImage

        //Array part

   byte[] value

   int dataType (values as per NDDataType_t)

   int [] dim

   + optional fields (timestamp, display, alarm, descriptor)

        //Image part

    int [] offset

    int [] fullDim

    int [] binning

    int [] reverse

        //Metadata part

    int uniqueId

    string [] attributeDescriptions

    int [] attributeSourceTypes

    string [] attributeSources

    NTVariantArray [] attributes

 Review of Nikolay's proposal [4].

Nikolay: AreaDetector is one of many different formats, so how might you define a structure that can encode many formats:

Following requirements by Tom Cobb (Diamond), NM’s amended recommended image carrying structure is the following;

struct  NTImage {

 

// image

Int width; // in pixels

Int height; // in pixels

Int depth; // bits per image pixel, the new definition

// color model

Int nChannels; // should be consistent with the color model

char colorModel[4]; // fourcc

char channelSeq[4]; // {'b', 'g', 'r'} or {'r', 'g', 'b'}

// buffer

int dataType; // extension of the IplImage structure

int size;// extension of the IplImage structure

char* data;

};

Where:

nChannels: number of channels (e.g. 1 –gray, 3 – rgb, etc), default: 1
depth: channel data type (e.g. pvUByte, pvByte, etc) default: pvUByte
width: image width in pixels, default: 0
height: image height in pixels, default: 0

NM: RGB or BGR would be primary format to keep performance up.

NM: If ColorModel is not defined, it would be grey or RGB [?]

NM: We can use CM to support special cases [xx], but if not there, again use grey.

NM: If a chanel seq is not defined, we can use BGR or grey.

NM: So it’s simple to check for the default mode [checking presence or absence of fields above].

NM: For depth assume bits per pixel.

NM: In BGR, depth gives one byte per channel-  24 bits.

Discussion on comments by Tom Cobb and NM’s reply.

NM: Made supportive remarks of Tom’s requirements.

NM: NTImage should be like a primary container, and so it should be able to encode

areaDetector, plus other formats. The primary formats encodable into NTImage, should

be those that may be in an areaDetector image.

AJ: Is skeptical that one could encode areaDetector simply into the structure NM presented above.

AJ: -1 on 2 image formats

AJ: Does MS think NTImage/AreaDetector is easy or hard to code for?

MS: After looking at some code, I found out how to interpret most of it. I would like to see upgrading areaDetector to support client images (eg JPEG).

MS: That should be relatively simple.

NM: Agrees in general, and points out that IPLimage [on which his structure above is based] is intended to do that and is well documented.

NM: On question of attributes, NTImage would include basically IPLimage (a generic structure), plus attributes (so including those things for areaDetector).

NM: - wants first to show how to encode an image for HDF5.

NM: - would like MS and DH to try putting NDarray into his [IPLImage] structure.

MS: How would you encode YUV444? [per Tom Cobb’s question]

NM: - wants Diamond to try to use his structure, using the image formats they use.

NM: asserts that exotic formats [like YUV444...] are not used. But good to demo BGR.

DH: We could translate areaDetector to IPLIMage, but it might be losing data and the

areaDetector metadata.

Area detector colorMode enum:

NDColorModeMono = 0,    /** Monochromatic image */
NDColorModeBayer = 1,   /** Bayer pattern image,
                           1 value per pixel but with color filter on detector */
NDColorModeRGB1 = 2,    /** RGB image with pixel color interleave,
                           data array is [3, NX, NY] */
NDColorModeRGB2 = 3,    /** RGB image with row color interleave,
                           data array is [NX, 3, NY]  */
NDColorModeRGB3 = 4,    /** RGB image with plane color interleave,
                           data array is [NX, NY, 3]  */
NDColorModeYUV444 = 5,  /** YUV image, 3 bytes encodes 1 RGB pixel */
NDColorModeYUV422 = 6,  /** YUV image, 4 bytes encodes 2 RGB pixel */
NDColorModeYUV411 = 7   /** YUV image, 6 bytes encodes 4 RGB pixels */

 

DH: Are you asking us to encode an areaDetector into an IPLImage?

NM: That’s not a good way to look at the issue: ndarray encodes a multi-dimentional image,

and in a v unique way. It also includes attributes (meta-data) which are not normally carried in image wrappers.

 

AJ: It is desirable to handle 3D images, particularly in the future.

NM: Use of AD in its 3D mode is not applicable to 3D images [!?]. It’s not applicable to tomography for instance.

OUTCOME: Agree that DH and MS should concentrate on tuning the design of NTImage as oriented toward areaDetector as now, rather than prototyping a general purpose data structure

per the IPLImage structure.

DH: Should NTImage be renamed to NTAreaDetector?

MS: If not renamed, can we better describe areadetector specifics like ColorMode.

DH: cm etc are defined in teh areaDetector doc, but more generally, ~ suggests simply

encoding the attributes right in the image part of the structure.

Description of what modifications should be made to the the normative types doc description.

AI on DH: Write to the mailing list his recommendations for help that may be given to a programmer to interpret the NTImage.

MEETING ENDS