NAMIC Wiki - User contributions [en]

OpenIGTLink/Discussion/NewCommands

2010-06-02T14:26:06Z

AlexanderSchaal: Reverted last change...

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
| align="left" | --
| align="left" | 8 bit unsigned
| align="left" | Reserved
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19:RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_LBMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''LBMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | --
| align="left" | 8 bit unsigned int
| align="left" | Reserved.
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color is defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | --
| align="left" | 8 bit unsigned
| align="left" | Reserved
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can be 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: 6D instrument (regular instrument), 3: 3D instrument (only tip of the instrument defined), 4: 5D instrument (tip and handle are defined, but not the normal vector)
|-
| align="left" | --
| align="left" | 8 bit unsigned
| align="left" | Reserved
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-06-02T14:06:13Z

AlexanderSchaal: /* Image data */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
| align="left" | --
| align="left" | 8 bit unsigned
| align="left" | Reserved
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19:RGB color 20:RGBA color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_LBMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''LBMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | --
| align="left" | 8 bit unsigned int
| align="left" | Reserved.
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color is defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | --
| align="left" | 8 bit unsigned
| align="left" | Reserved
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can be 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: 6D instrument (regular instrument), 3: 3D instrument (only tip of the instrument defined), 4: 5D instrument (tip and handle are defined, but not the normal vector)
|-
| align="left" | --
| align="left" | 8 bit unsigned
| align="left" | Reserved
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-20T08:42:34Z

AlexanderSchaal: /* Point or fiducal data */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_LBMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''LBMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color is defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can be 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: 6D instrument (regular instrument), 3: 3D instrument (only tip of the instrument defined), 4: 5D instrument (tip and handle are defined, but not the normal vector)
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-20T08:42:15Z

AlexanderSchaal: /* Point or fiducal data */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_LBMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''LBMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color is defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: 6D instrument (regular instrument), 3: 3D instrument (only tip of the instrument defined), 4: 5D instrument (tip and handle are defined, but not the normal vector)
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-16T14:10:30Z

AlexanderSchaal: /* Tracking data */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_LBMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''LBMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color is defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: 6D instrument (regular instrument), 3: 3D instrument (only tip of the instrument defined), 4: 5D instrument (tip and handle are defined, but not the normal vector)
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-16T14:09:42Z

AlexanderSchaal: /* Tracking data */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_LBMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''LBMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color is defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: 6D instrument (regular instrument), 3: 3D instrument (only tip of the instrument defined), 4: 5D (instrument with tip and handle, but without normal vector)
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-16T13:38:08Z

AlexanderSchaal: /* Label map / Voxel objects */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_LBMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''LBMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color is defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: regular instrument with tip and handle, 3: instrument only with tip defined, 4: instrument with tip and handle, but without normal vector
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-16T12:38:43Z

AlexanderSchaal: /* Tracking data */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_STRCMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_STRCMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color id defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: regular instrument with tip and handle, 3: instrument only with tip defined, 4: instrument with tip and handle, but without normal vector
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-16T09:36:52Z

AlexanderSchaal: /* Comments */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_STRCMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_STRCMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color id defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Resolution
| align="left" | 32 bit unsigned
| align="left" | Minimum time between two frames. Use 0 for as fast as possible. If e.g. 50 ms is specified, the maximum update rate will be 20 Hz.
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-16T07:27:57Z

AlexanderSchaal: /* Label map / Voxel objects */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_STRCMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_STRCMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color id defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-16T07:11:27Z

AlexanderSchaal: /* Label map / Voxel objects */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_STRUMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_STRUMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color id defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE), bounding box of the structure(s)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-16T07:08:55Z

AlexanderSchaal: /* Voxel objects */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Label map / Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects or a label map, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available structures. I suggest a GET_STRUMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_STRUMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | Label
| align="left" | 8 bit unsigned int
| align="left" | Label of structure (0 if unused)
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA (0 0 0 0 if no color id defined)
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused. Can be empty if n/a.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-15T07:44:27Z

AlexanderSchaal: /* Voxel objects */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available voxelobjects. I suggest a GET_VOXMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_VOXMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-15T07:31:29Z

AlexanderSchaal:

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available voxelobjects. I suggest a GET_VOXMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_VOXMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTS_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTS_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTS_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-15T07:30:30Z

AlexanderSchaal:

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available voxelobjects. I suggest a GET_VOXMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_VOXMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RTG_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RTG_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RTG_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-15T07:23:59Z

AlexanderSchaal:

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Voxel objects =

Voxel objects can be tumors, segmented structures likes eyes, etc. They are typically much smaller than the owning slicesets and can be overlayed on the sliceset data.

To retreive voxel objects, GET_IMAGE / IMAGE can be used. But the client should be able to get a list of available voxelobjects. I suggest a GET_VOXMETA message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_VOXMETA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Voxel objects from different slicesets can be sent if slicesets are fused.
|-
|}

== Comments ==

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RET_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RET_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RET_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-14T07:47:08Z

AlexanderSchaal:

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RET_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RET_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RET_TDATA. The real data, which is pushed is called TDATA now, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-14T07:44:22Z

AlexanderSchaal: /* Comments */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=
*This is a log page for the discussion about new OpenIGTLink messages introduced in protocol version 2.

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

''Junichi:''
* I totally agree. The STATUS message was originally designed for hardware e.g. biopsy robot and may not fit to other applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STT_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STP_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr STT_TDATA and STP_TDATA:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''RET_TDATA'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

''Junichi:''
* Sounds good idea to specify what happens if empty device name is specified.
* Yes, I think we need to address how to return status.
** It's a nice idea to define status messages.
** I would suggest more consistent naming convention
***For starting data push START_<device name>
***For stopping data push STOP_<device name>
***For status STATUS_<device name>
** By the way, 'START_' 'STOP_' and 'STATUS_' seem too long, because we only have 12 bytes for device name. Would be nice to have 3-letter codes ex. 'STT_', 'STP_', 'RET_'. (this improves the consistency with 'GET_')

''Alexander:''
* I totally agree. I have changed the messages above to STT_TDATA, STP_TDATA and the returning status to RET_TDATA. The real data, which is pushed is still called TDATA, see below.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T09:16:16Z

AlexanderSchaal: /* Image data */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.

To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. The following is copied from [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]]. I have removed the version and reserved fields - or why do we need a version in the header and in the body?
* GET_COLORT has no parameter except the device name field in the header.
* See below the COLORTABLE message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | I
| align="left" | 8-bit unsigned int
| align="left" | Index Type (3:uint8 5:uint16)
|-
| align="left" | M
| align="left" | 8-bit unsigned int
| align="left" | Map Type (3:uint8 5:uint16 19: RGB color)
|-
| align="left" | TABLE
| align="left" | Array of 8-bit unsigned int
| align="left" | Color index table
|-
|}

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

''Alexander'': I have added the COLORTABLE message above.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr START_TDPUSH and STOP_TDPUSH (means same type string except START and STOP):

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T08:58:34Z

AlexanderSchaal: /* Comments */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-op landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr START_TDPUSH and STOP_TDPUSH (means same type string except START and STOP):

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T08:55:02Z

AlexanderSchaal: /* Comments */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" messages:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message can be sent.
** After stopping the push, it might be possible that some TDATA messages are still in the pipeline. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about a typical answer message forr START_TDPUSH and STOP_TDPUSH (means same type string except START and STOP):

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDPUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the query-answer-mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE/IMAGE, etc.

What do you think about that?

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T08:36:25Z

AlexanderSchaal: /* Comments */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" message (only one to make it easier):

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Action
| align="left" | 8 bit unsigned
| align="left" | 1: start push 2: stop push
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent. (not included if action = 2)
|-
| align="left" | Coordinate system name
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system. (not included if action = 2)
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message will be sent.
** After STOP_TDATA it might be possible to receive the last TDATA message until the sever really stops. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about an additional message, which shall be sent as answer after GET_TDATAPSH:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the "GET_<device name> -- Answer" mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE -- IMAGE.

What do you think about that?

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T08:35:33Z

AlexanderSchaal: /* Comments */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We should dismiss START_PUSH and STOP_PUSH due to
** type and device name must be specified,
** inconsistencies with GET_<device type>,
** we need a data specific body.
See below the "new" message (only one to make it easier):

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Action
| align="left" | 8 bit unsigned
| align="left" | 1: start push 2: stop push
|-
| align="left" | Max Resolution (not included if action = 2)
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Coordinate system name (not included if action = 2)
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system.
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message will be sent.
** After STOP_TDATA it might be possible to receive the last TDATA message until the sever really stops. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about an additional message, which shall be sent as answer after GET_TDATAPSH:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the "GET_<device name> -- Answer" mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE -- IMAGE.

What do you think about that?

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T08:33:50Z

AlexanderSchaal: /* Comments */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We can dismiss START_PUSH and STOP_PUSH due to
** device type and name must be specified
** inconsistencies with GET_<device type>
** we need a data specific body
See below the "new" message (only one to make it easier):

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Action
| align="left" | 8 bit unsigned
| align="left" | 1: start push 2: stop push
|-
| align="left" | Max Resolution (not included if action = 2)
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Coordinate system name (not included if action = 2)
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system.
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message will be sent.
** After STOP_TDATA it might be possible to receive the last TDATA message until the sever really stops. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about an additional message, which shall be sent as answer after GET_TDATAPSH:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the "GET_<device name> -- Answer" mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE -- IMAGE.

What do you think about that?

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T08:33:14Z

AlexanderSchaal: /* Comments */

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I think about using '''always''' the original answer message type with appropriate body data on success and body size 0 if an error has occurred, because in almost every case the important information is "data not available" and not "reason abc".

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We can dismiss START_PUSH and STOP_PUSH due to
** device type and name must be specified
** inconsistencies with GET_<device type>
** we need a data specific body
See below the "new" message (only one to make it easier):

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Action
| align="left" | 8 bit unsigned
| align="left" | 1: start push 2: stop push
|-
| align="left" | Max Resolution (not included if action = 2)
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Coordinate system name (not included if action = 2)
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system.
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message will be sent.
** After STOP_TDATA it might be possible to receive the last TDATA message until the sever really stops. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about an additional message, which shall be sent as answer after GET_TDATAPSH:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the "GET_<device name> -- Answer" mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE -- IMAGE.

What do you think about that?

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T08:29:08Z

AlexanderSchaal:

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include device type and name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the device type. Maybe the interface needs a STATUS 2.0 message? However, I can live with returning an answer with body size 0, because in almost every case the important information is "data not available" and not "reason abc". --> I think about using '''always''' the original answer message type with appropriate body data on success and body size 0 if an error has occurred.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

''Alexander:''
* Agreed. We can dismiss START_PUSH and STOP_PUSH due to
** device type and name must be specified
** inconsistencies with GET_<device type>
** we need a data specific body
See below the "new" message (only one to make it easier):

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''GET_TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Action
| align="left" | 8 bit unsigned
| align="left" | 1: start push 2: stop push
|-
| align="left" | Max Resolution (not included if action = 2)
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Coordinate system name (not included if action = 2)
| align="left" | char[20]
| align="left" | Coordinate system to use. Can be empty for default coordinate system.
|-
|}

''Alexander:''
* After thinking about the device name, the following can be specified:
** If device name is empty, all visible trackers/instruments will be pushed.
** If device name is not empty, only the appropriate tracker/instrument will be pushed.
* We need something to indicate, that pushing is started or stopped. Examples:
** An error could occur, e.g. the device name = instrument name is not valid, no TDATA message will be sent.
** After STOP_TDATA it might be possible to receive the last TDATA message until the sever really stops. The stop should be acknowledged so the client can rely on not getting TDATA messages anymore.
* This could be done by a STATUS message, but as already wrote, the STATUS message does not have enough fields. I think about an additional message, which shall be sent as answer after GET_TDATAPSH:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TDATAPSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Status
| align="left" | 8 bit unsigned
| align="left" | 0: Success 1: Error
|-
|}

''Alexander:''
* Using the "GET_<device name> -- Answer" mechanism makes starting/stopping the push consistent to e.g. GET_IMAGE -- IMAGE.

What do you think about that?

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

''Alexander'':
* I think we don't need the status type. But maybe we define a TDATA in v3 after collecting some feedback from different users. I could think about other fields like diameter of instruments, etc. But for now I would like to keep TDATA as simple as possible.
* Ok, starting another trackingdata push with the same device name will implictly stop the first one. But if another device name is used, a second push will be started.

OpenIGTLink/Discussion/NewCommands

2010-04-13T07:33:19Z

AlexanderSchaal:

[[OpenIGTLink | << OpenIGTLink]]

=Objective=

=What's new in OpenIGTLink Protocol version 2 and Library Version 2=
*Better support for IGS Systems
**New IGS specific message types
*Matlab interface support
**Remote Matlab command execution
**Matlab interface library
**IGS support in Matlab
*Other new messages
**Associative Array message
**NIfTI support ?

= Timeline for V.2 Protocol Release=
*Events:
*June 20 - June 26: [http://www.na-mic.org/Wiki/index.php/2010_Summer_Project_Week NA-MIC Summer Project Week in Boston]
**June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
**June 22, 16:00 - : OpenIGTLink User Group Meeting
**Release v.2.0!!
*Notes:
**Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left" style="background:#e0e0e0;" | Week
| align="left" style="background:#eeeeee;" | 4/11 - 4/17
| align="left" style="background:#eeeeee;" | 4/18 - 4/24
| align="left" style="background:#eeeeee;" | 4/25 - 5/1
| align="left" style="background:#eeeeee;" | 5/2 - 5/8
| align="left" style="background:#eeeeee;" | 5/9 - 5/15
| align="left" style="background:#eeeeee;" | 5/16 - 5/22
| align="left" style="background:#eeeeee;" | 5/23 - 5/29
| align="left" style="background:#eeeeee;" | 5/30 - 6/5
| align="left" style="background:#eeeeee;" | 6/6 - 6/12
| align="left" style="background:#eeeeee;" | 6/13 - 6/19
| align="left" style="background:#eeeeee;" | 6/20 - 6/26
|-
| align="left" | Events
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |NA-MIC Project Week
|-
| align="left" | Protocol Draft [IGS]
| align="left" style="background:#a0a0a0;"|Review
| align="left" style="background:#a0a0a0;"|Review, Freeze
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Protocol Draft [other]
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Draft
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" style="background:#a0a0a0;"| Review
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Review, Freeze
|-
| align="left" | IGS System (Alexander)
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Dev
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" style="background:#a0a0a0;"|Test
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | Library (Junichi)
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Dev (IGS)
| align="left" style="background:#a0a0a0;"| Test (IGS)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Dev (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" style="background:#a0a0a0;"| Test (other)
| align="left" |
|-
| align="left" | Matlab IF (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
| align="left" |
| align="left" |
|-
| align="left" | 3D Slicer (OIGTL IF) (Junichi)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
| align="left" | 3D Slicer (IGS Module) (Haiying)
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" |
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Dev
| align="left" style="background:#a0a0a0;"| Test
| align="left" style="background:#a0a0a0;"| Test
| align="left" |
|-
|}

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

''Junichi:'' It makes sense to leave the error name as it is. Putting device type into the device name is a bit confusing,if there is a device with a name same as device type. I'm start thinking that your initial idea (message with body size 0) makes sense.

''Alexander:''
* Ok, let's define if item(s) is/are not available, an answer with body size 0 shall be returned. In this way it's easy to include type and device name.
* If something went wrong during processing the query, a STATUS message could be returned, but as you already wrote, a field is missing for specifying the type name. Maybe the interface needs a STATUS 2.0 message? However, I can live with returning an answer with body size 0, because in almost every case the important information is "data not available" and not "reason abc". --> I think about using '''always''' the original answer message type with appropriate body data on success and body size 0 if an error has occurred.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

''Junichi'': Sounds good.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

''Junichi:''
* I would add device type field in the START_PUSH and STOP_PUSH. The device name field cannot be used for specifying device type, because data source should be specified by a pair of device name and device type in OpenIGTLink.
* But I, at the same time, start thinking that your original idea makes sense. A device type can be START_<device type> / STOP_<device type> instead of START_PUSH / STOP_PULL, because this is consistent with what we do with GET_<device type> message.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

''Junichi'':
* I haven't defined status types... it can be a bit array like: bit 0: registered or not; bit 1: line-of-site error; ....
** Do you think it is useful? if not, we can omit it.
* I agree that we need a way to specify coordinate system. It's good idea to have data specific field in the START_PUSH.
* I agree with the last comment. I would say one TRACKINGDATA push for each device name, because multiple data sources may exist. The coordinate system can be overwritten by another START_PUSH message with the same device name and type.

OpenIGTLink/Discussion/NewCommands

2010-04-09T11:42:41Z

AlexanderSchaal: /* Comments */

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be answered by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

OpenIGTLink/Discussion/NewCommands

2010-04-09T11:38:41Z

AlexanderSchaal:

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be returned by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''START_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Max Resolution
| align="left" | 32 bit unsigned
| align="left" | Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''STOP_PUSH'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Data specific arguments
| align="left" | char[bodysize - 4]
| align="left" | Can be everything, depends on the specific data type.
|-
|}

==Comments==
''Junichi:''
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

''Alexander:''
* Yes, the device name field of the header could be e.g. TRACKINGDATA.
* Good idea about the maximum resolution!
I have added two tables above.

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
''Junichi'':
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

''Alexander'':
*What status types can be specified?
*In the case the coordinate system is not valid, a STATUS message should be returned.
*Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
*Due to consistency, I think we should add this field to STOP_PUSH, too.
*We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?

OpenIGTLink/Discussion/NewCommands

2010-04-09T11:20:18Z

AlexanderSchaal:

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be returned by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name/Description
| align="left" | char[64]
| align="left" | Name or description of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Timestamp
| align="left" | 64 bit unsigned
| align="left" | Scan time, see [[OpenIGTLink/Timestamp]]
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
''Junichi'':
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

''Alexander'':
*Agree to your first point. I have adapted the table above.
*Agree to your second point.
*Regarding your third point: let's call it "Name or description".

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''POINT'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the point
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", ...
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X,Y,Z
| align="left" | 32 bit float
| align="left" | Coordinate of the point
|-
| align="left" | Radius
| align="left" | 32 bit float
| align="left" | Radius of the point, can be 0.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

{| border="0" cellpadding="5" cellspacing="0" align="center"
|-
| '''TRAJECTORY'''
|-
|}

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Name or description of the trajectory.
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Entry point of the trajectory
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Diameter of trajectory, can b 0
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

==Comments==
''Junichi:''
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

''Alexander:''
*Yes, points can have a diameter/radius.
*Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
*If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

==Comments==
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

OpenIGTLink/Discussion/NewCommands

2010-04-09T10:55:24Z

AlexanderSchaal:

= Common GET_LIST message ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

==Comments==
''Junichi:'' I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

''Alexander:'' Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

==Comments==
''Junichi:'' The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:
*Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
*Host B receives the GET_IMAGE message, but it does not have such image.
*Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".
Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

''Alexander:'' Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:
*A GET message shall be returned by exactly one answer message.
*The answer messages shall be returned in the same sequence as the GET messages were sent.
However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Pretty name of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Day
| align="left" | 8 bit unsigned
| align="left" | Scan day
|-
| align="left" | Month
| align="left" | 8 bit unsigned
| align="left" | Scan month
|-
| align="left" | Year
| align="left" | 16 bit unsigned
| align="left" | Scan year
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

==Comments==
*I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see [[OpenIGTLink/Timestamp]].)
*If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
*I would call the first field "Image description"

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT is used to get all the point data. It has no parameters. The answer is a POINT message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Pretty name of the point data like "Fiducal behind left ear"
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: single point, 2: trajectory with only entry point, 4: trajectory with only target point, 6: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Coordinate of the single point or entry point of the trajectory (which consists of an entry and a target point)
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory. If the trajectory has length 0, it is equal to the entry point. For single points the coordinates are 0,0,0
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Always 0 for single points, can be 0 for trajectories.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.

==Comments==
*For diameter, I think It's OK to allow value greater than 0 for single point.
*Do we really need group name?
*Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

= Start / stop push =

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

==Comments==
*possible fields in START_PUSH message:
**device type (TRACKINGDATA, TRANSFORM, IMAGE)
**maximum time resolution (Hz)

= Tracking data =

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.
* Specifing that something is not valid/visible anymore (could be done by 0-matrix)
* Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
* Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
* Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[20]
| align="left" | Name (=Id) of the instrument/tracker
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
|-
| align="left" | Matrix
| align="left" | 32 bit float
| align="left" | 12 values like in TRANSFORM message
|-
|}

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: '''while(true) { recv(trackingdata); updateView(trackingdata); }'''

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body
of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without
explicitly specifing the camera coordinate system.

==Comments==
*How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

OpenIGTLink/Discussion/NewCommands

2010-04-07T14:14:32Z

AlexanderSchaal: /* Tracking data */

OpenIGTLink/Discussion/NewCommands

2010-04-07T14:12:53Z

AlexanderSchaal: /* Start / stop push */

OpenIGTLink/Discussion/NewCommands

2010-04-07T14:11:29Z

AlexanderSchaal: /* Point or fiducal data */

OpenIGTLink/Discussion/NewCommands

2010-04-07T14:09:32Z

AlexanderSchaal: /* GET messages */

OpenIGTLink/Discussion/NewCommands

2010-04-07T14:08:30Z

AlexanderSchaal: /* Common GET_LIST message ? */

OpenIGTLink/Discussion/NewCommands

2010-04-07T14:07:27Z

AlexanderSchaal: /* Common GET_LIST message ? */

OpenIGTLink/Discussion/NewCommands

2010-04-07T14:06:44Z

AlexanderSchaal: /* Common GET_LIST message ? */

OpenIGTLink/Discussion/NewCommands

2010-04-07T14:04:03Z

AlexanderSchaal:

OpenIGTLink/Discussion/NewCommands

2010-04-07T11:20:20Z

AlexanderSchaal:

OpenIGTLink/Discussion/NewCommands

2010-04-07T11:19:54Z

AlexanderSchaal:

= Common get list ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so it can they used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application would like to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once, which returns a list of meta data.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

= GET messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer might be 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Pretty name of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Day
| align="left" | 8 bit unsigned
| align="left" | Scan day
|-
| align="left" | Month
| align="left" | 8 bit unsigned
| align="left" | Scan month
|-
| align="left" | Year
| align="left" | 16 bit unsigned
| align="left" | Scan year
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT is used to get all the point data. It has no parameters. The answer is a POINTS message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Pretty name of the point data like "Fiducal behind left ear"
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: single point, 2: trajectory with only entry point, 4: trajectory with only target point, 6: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Coordinate of the single point or entry point of the trajectory (which consists of an entry and a target point)
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory. If the trajectory has length 0, it is equal to the entry point. For single points the coordinates are 0,0,0
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Always 0 for single points, can be 0 for trajectories.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.

= Start / stop push =

START_PUSH + STOP_PUSH

== Tracking data ==

TRACKINGDATA

OpenIGTLink/Discussion/NewCommands

2010-04-07T11:19:31Z

AlexanderSchaal:

= Common get list ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so it can they used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application would like to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once, which returns a list of meta data.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

= Errors in GET-messages =

Get messages are not well defined yet. The following things should be fixed:
* If 0 items are available, the bodysize of the answer might be 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Pretty name of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Day
| align="left" | 8 bit unsigned
| align="left" | Scan day
|-
| align="left" | Month
| align="left" | 8 bit unsigned
| align="left" | Scan month
|-
| align="left" | Year
| align="left" | 16 bit unsigned
| align="left" | Scan year
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT is used to get all the point data. It has no parameters. The answer is a POINTS message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Pretty name of the point data like "Fiducal behind left ear"
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: single point, 2: trajectory with only entry point, 4: trajectory with only target point, 6: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Coordinate of the single point or entry point of the trajectory (which consists of an entry and a target point)
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory. If the trajectory has length 0, it is equal to the entry point. For single points the coordinates are 0,0,0
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Always 0 for single points, can be 0 for trajectories.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.

= Start / stop push =

START_PUSH + STOP_PUSH

== Tracking data ==

TRACKINGDATA

OpenIGTLink/Discussion/NewCommands

2010-04-07T11:16:22Z

AlexanderSchaal:

= Common get list ? =

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so it can they used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

'''Well, I don't think this will help much in practice.'''

''Example:'' an application would like to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA.
It would be easier to get all the meta data at once, which returns a list of meta data.

''Next example:'' all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. '''The number is bodySize/itemSize.''' Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

= Image data =

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Pretty name of the image
|-
| align="left" | Id
| align="left" | char[20]
| align="left" | Id to query the IMAGE and COLORT
|-
| align="left" | Modality
| align="left" | char[32]
| align="left" | String which specifies the modality
|-
| align="left" | Patient name
| align="left" | char[64]
| align="left" | Name of the patient
|-
| align="left" | Patient id
| align="left" | char[64]
| align="left" | Id of the patient
|-
| align="left" | Day
| align="left" | 8 bit unsigned
| align="left" | Scan day
|-
| align="left" | Month
| align="left" | 8 bit unsigned
| align="left" | Scan month
|-
| align="left" | Year
| align="left" | 16 bit unsigned
| align="left" | Scan year
|-
| align="left" | RI, RJ, RK
| align="left" | 16 bit unsigned
| align="left" | Number of pixels in each direction (same as in IMAGE)
|-
| align="left" | S
| align="left" | 8 bit unsigned
| align="left" | Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See [[OpenIGTLink/ProtocolV2|Standard OpenIGTLink Protocol description -- Version 2 draft]] for details.

= Point or fiducal data =

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT is used to get all the point data. It has no parameters. The answer is a POINTS message:

{| border="1" cellpadding="5" cellspacing="0" align="center"
|-
| align="left style="background:#e0e0e0;" | Data
| align="left style="background:#e0e0e0;" | Type
| align="left style="background:#e0e0e0;" | Description
|-
| align="left" | Name
| align="left" | char[64]
| align="left" | Pretty name of the point data like "Fiducal behind left ear"
|-
| align="left" | Group name
| align="left" | char[32]
| align="left" | Can be "Labeled Point", "Landmark", "Fiducal", "Trajectory", ...
|-
| align="left" | Type
| align="left" | 8 bit unsigned
| align="left" | 1: single point, 2: trajectory with only entry point, 4: trajectory with only target point, 6: trajectory with entry and target point
|-
| align="left" | R,G,B,A
| align="left" | 8 bit unsigned
| align="left" | Color in RGBA
|-
| align="left" | X1,Y1,Z1
| align="left" | 32 bit float
| align="left" | Coordinate of the single point or entry point of the trajectory (which consists of an entry and a target point)
|-
| align="left" | X2,Y2,Z2
| align="left" | 32 bit float
| align="left" | Target point of a trajectory. If the trajectory has length 0, it is equal to the entry point. For single points the coordinates are 0,0,0
|-
| align="left" | Diameter
| align="left" | 32 bit float
| align="left" | Always 0 for single points, can be 0 for trajectories.
|-
| align="left" | Owner image
| align="left" | char[20]
| align="left" | Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.
|-
|}

* More than one item can be transmitted. The number is bodySize/itemSize.

= Start / stop push =

START_PUSH + STOP_PUSH

== Tracking data ==

TRACKINGDATA

OpenIGTLink/Discussion/NewCommands

2010-04-07T11:00:31Z

AlexanderSchaal:

OpenIGTLink/Discussion/NewCommands

2010-04-07T09:40:52Z

AlexanderSchaal: /* Common get list ? */

OpenIGTLink/Discussion/NewCommands

2010-04-07T09:40:15Z

AlexanderSchaal:

OpenIGTLink/Discussion/NewCommands

2010-04-07T09:01:45Z

AlexanderSchaal:

OpenIGTLink/Discussion

2010-04-06T15:08:42Z

AlexanderSchaal: /* New commands */

[[OpenIGTLink | << OpenIGTLink]]

=Requests, TODOs=
*[IGTL]Broadcasting (The Open IGT Link Library)
*[Slicer]Fix problem in transform matrix

=Project Week 2008 @ MIT=
*The main page for this project is [[2008_Summer_Project_Week]]

==Attendee==
*Junichi Tokuda
*Haiying Liu

==Agenda==

===Open IGT Link -- new requirements===
*requirements for new software
**BrainLab VVLink
**Prostate
**Neuro

===Slicer 3 Open IGT Link Module -- What is next?===
*Outgoing data via MRML
*GUI Interface Design
*API Design to use OpenIGTLink module from other modules

= New commands =

Discussion page about [[OpenIGTLink/Discussion/NewCommands| new commands]] based on version 1 of OpenIGTLink. It is not decided yet, if (parts of) these commands are integrated to one of the next official
interface versions.

= Open questions =
# What if the application receives hundreds of "position" packets? Should we have packet types? -keep only the last (position) - all important (command) - high priority (emergency) 
# Priority
# Authentication (other than Unique Name) -- SSL
# Compression?
# Duplicate messages
# What about the check of the header ?? -- by Dr. Gregory Fischer @ WPI

= Other Resources =
* [http://www.bioimagesuite.org/public/VVLink.html BioImage Suite VVLink Tool] (uses TCL/TK object streaming)
* [[OpenIGTLink/Meetings | Meetings concerning Open IGT Link ]]

OpenIGTLink/Discussion/NewCommands

2010-04-06T09:20:39Z

AlexanderSchaal:

This page will be edited soon. The topics will be:

= Push or pull? =

...

= Image data =

IMAGE + COLORTABLE + METADATA

= Point or fiducal data =

POINTS

= Start / stop push =

START_PUSH + STOP_PUSH

== Tracking data ==

TRACKINGDATA

OpenIGTLink/Discussion

2010-04-06T09:17:42Z

AlexanderSchaal: /* New commands */

[[OpenIGTLink | << OpenIGTLink]]

=Requests, TODOs=
*[IGTL]Broadcasting (The Open IGT Link Library)
*[Slicer]Fix problem in transform matrix

=Project Week 2008 @ MIT=
*The main page for this project is [[2008_Summer_Project_Week]]

==Attendee==
*Junichi Tokuda
*Haiying Liu

==Agenda==

===Open IGT Link -- new requirements===
*requirements for new software
**BrainLab VVLink
**Prostate
**Neuro

===Slicer 3 Open IGT Link Module -- What is next?===
*Outgoing data via MRML
*GUI Interface Design
*API Design to use OpenIGTLink module from other modules

= New commands =

Discussion page about [[OpenIGTLink/Discussion/NewCommands| new commands]] based on version 1 of OpenIGTLink. It is not clear yet, if (parts of) these commands are integrated to a the next official
interface version.

= Open questions =
# What if the application receives hundreds of "position" packets? Should we have packet types? -keep only the last (position) - all important (command) - high priority (emergency) 
# Priority
# Authentication (other than Unique Name) -- SSL
# Compression?
# Duplicate messages
# What about the check of the header ?? -- by Dr. Gregory Fischer @ WPI

= Other Resources =
* [http://www.bioimagesuite.org/public/VVLink.html BioImage Suite VVLink Tool] (uses TCL/TK object streaming)
* [[OpenIGTLink/Meetings | Meetings concerning Open IGT Link ]]