The focus of this release has been to address a number of user feature requests.

-  Allow all segments to be displayed at once in the Segment Editor, with the ability to export the displayed information. -- Requested by Glenn at South Eastern Sydney and Illawarra Area Health Services.

- Add percentages to Field Statistics and Message Statistics. -- Requested by Glenn at South Eastern Sydney and Illawarra Area Health Services.

- Allow searching for a value in a repeating segment or repeating field. For example, a repeating segment query like NK1[*]-3 = 'Mother', or a repeating field query like PID-3[*].5='ORG1'. Currently this type of query is only supported in the Search Dialog (Cntrl-F) in the Message Editor. In an upcoming release, this will also be added to the HL7 SQL feature. -- Requested by Paul at T-Systems.

- Export Field Statistics, Summary Statistics, and HL7 SQL results using native Excel data types, rather than as strings. -- Requested by Paul at T-Systems.

- Add Regular Expression help to Search Dialog-- Requested by Paul at T-Systems.

- Add support for automatically detecting and opening Epic message files. - Requested by Dave at Philips Medical.

- Display Date/Times in  MM/DD/YYYY HH:mm:ss format HL7 SQL data grid, with option to display in HL7 format. -- Requested by Marnie Hare at Inner Harbour Software

- Extend the capabilities of DataGenerator.GetRandomValue() in the Custom Code feature to handle user defined data files -- Requested by John at Scientific Technology Corporation.

Sample Code (cut and paste this into the Custom Code Editor):

// create a data generator
private static DataGenerator _dataGenerator = DataGenerator.Create("USA");
public override void Run()
{
  // this samples uses  the Provider.txt file, however; any ^ separated file can be used
  UserDictionary providers = _dataGenerator.GetUserDictionary("Providers.txt");
 
  if(providers==null){
    Log(Severity.Error,"Could not find requested dictionary");
    return;
  }
 
  for(int i=0;i<100;++i) // create 100 sample messages
  {
    // create a message
    Hl7Message message = new Hl7Message();
    // create a Z-Segment
    Segment seg = new Segment("ZZZ");
    // add the Z-Segment to the message
    message.Append(seg);
    string[] values = providers.GetRandomRow(); // get a random provider
    int field=0;
    foreach(string value in values)
    {
      seg[++field]=value; // add values to the Z-Segment
    }
    // save the message
    this.SaveMessage(message.ToString(),"User Dictionary Sample");
  }
  Cancel = true; // ensure we only run once
}

 Bug Fixes

- Chinese characters not displaying correctly for first message

- Incorrect SQL text sent (sometimes) to database engine in the Sql Loader tool

- Performance improvements to Sql Loader tool

- Sql Loader displays no results if the order of the columns in the select statement is changed.

- Advanced Statistics not returning the correct results for repeated fields (sometimes)

- Fix Modal issue with MessageFraming dialog

- Fix disappearing column in the Segment Editor

- More intelligent parsing of query in HL7 SQL editor

This is a minor bug fix release with 3 very minor changes.

- Allow lower case segment names

- Do not automatically try to restore remote mounted directories because of potentially long delays if the remote directory becomes unavailable.

- Do not show the results tab when the user is just compiling, or saving code.

New Advanced Statistics Tool

This tool allows calculation of statistics across repeating fields and repeating segments. It also provides filtering based on the contents of any field within the repeating segment. For example, if you want to find the statistics of OBX-5 across all repeating OBX segments where OBX-3 has a value of TNMODE, then this tool will do the trick. (Thanks to Paul Coyne from TSystems for this feature suggestion)

As another example, the figure below shows how to get the statistics of all allergy descriptions across all messages for the first 2 AL1 segments in the message.

Here is a description of the various settings in this dialog.

1. HL7 Path – this will be the path to the field for which the statistics will be calculated. The path is set to the currently selected path in the Message Editor window at the time the Advanced Statistics button is clicked in the toolbar. This path can be changed at any time by clicking on a field within the Message Editor. There is no need to close the Advanced Statistics dialog to do this.

2. Component or Field – sets whether the statistics will be calculated for the specified component, or the entire field.

3. Include Segment Repeats – sets whether or not all matching segments are included in the statistics. If not selected, only segments matching the specified segment repeat will be included in the statistics calculation.

4. Include Field Repeats – sets whether or not all repeating fields are included in the statistics. If not selected, only the specified field repeat will be included in the statistics calculation.

5. Only include repeated segments that match the following criteria – sets whether segment filter is enabled or not.

6. Sets the Component and Field location that will be used in the matching criteria. If component is set to 0, then the entire field is used in the matching criteria.

7. A comma separated list – sets the matching criteria that the field specified in (6) must match for the repeated segment to be included in the statistics calculation.

    

Official Support for Windows 7 and Windows 2008 x86 and x64

• HL7Spy has now been validated on all the following Microsoft Operating Systems: Windows XP, Vista, Vista, Windows 2003, Windows Windows 2008, and Windows 7.

Many Small Improvements

• It might be hard to believe, but HL7Spy now performs searches and queries even faster than before.

• Connect/Disconnect the TCP/IP connection in the HL7 Send window. To access this functionality right click on the connection status indicator to the left of the “All” button.

• Better handing of large files in the OpenFolder dialog.

• Better handing of localhost when IPv6 is installed.

New Custom Code Features:

- Access to the raw message bytes from a new Message property.

- Two new SaveMessage methods:

   Save(IMessageData message, string collectionName) ;

   Save(IEnumerable<IMessageData> message, string collectionName) ;

These methods will allow you to create new MessageCollection tabs from messages in the current tab.

For example, the following custom function creates a new tab containing the last ADT^A08 message received for each unique patient identifier found in a collection of messages. (This example was a customer request to aid in the calculation of statistics of ADT^A08 messages. The requirement was to only include a single ADT^A08 message per patient identifier)

string _fileName = @"c:\_test.txt";

Dictionary<string,Patient> _data = new Dictionary<string,Patient>(10000);

public class Patient

{

  public Patient(string patientId)

  {

    PatientId = patientId;

  }

  public IMessageData Message;

  public string PatientId;

}

public override void Run() {

  // Get an HL7 Message in parsed format

  Hl7Message message = GetParsedMessage();

  MSH msh = message.GetFirstSegment<MSH>();

  if(msh==null || msh.MessageType_09.TriggerEvent_02.Value!="A08")

    return; // not an ADT^A08 message

  PID pid = message.GetFirstSegment<PID>();

  if(pid==null)

    return; // no pid segment

    string patientId = pid.PatientIdentifierList_03.First.Value;

  Patient pat;

  if(!_data.TryGetValue(patientId,out pat))

  {

     pat = new Patient(patientId);

    _data.Add(patientId,pat);

  }

  pat.Message = Message;

 }

// Called once after the last message has been processed.

// It is a good place to perform cleanup and to report information.

// Always called from the UI thread.

public override void OnFinish()

{

  // Save all the messages collected in an new Message Tab

  SaveMessage(_data.Values.Select(p=>p.Message),"Last A08");

}

In addition to some minor bug fixes, this release contains the following enhancements:

• Socket timeouts for both HL7 Send and Receive
  
• An option to the ‘Save As’ dialog to save batches of messages using the Batch File Structured as defined in the HL7 Standard V2.6,  Chapter 2, Section 2.10.3.1.

 Both of these features were requested by our user community. Thank you for your support and great ideas.

New HL7 Query for HL7Spy Professional

The HL7 Query window provides a SQL like interface for querying an HL7 Message collections. This can be particularly useful for finding messages that match a set of very specific criteria. For example, the figure below shows the results of executing the following query:

select MSH-7, MSH-9.*, PID-3, PID-5.*, PV1-2

    where PV1-1 not null and PV1-2 NOT IN ('I','E','P')

This query displays a row for each message found that contains a PV1 segment, and where PV1-3 is not one of 'I', 'E', or 'P'. In the case of the sample data that ships with HL7Spy, this query returns 9084 rows.

For each row in the matching set, the HL7 Paths MSH-7, MSH-9.*, PID-3 and PID-5.* are displayed. Note the aliasing of MSH-7 to 'Date/Time' (see below for a discussion of HL7 Paths).

Clicking on one of the rows in the data grid will cause the message associated with that row to be displayed in the message editor.

The supported query specification is as follows:

SELECT [ TOP ( integer ) ] <HL7Path_list> [where < search_condition >]

<HL7Path_list> ::= HL7Path [ [ AS ] column_alias ][,...n]

< search_condition > ::=

{ [ NOT ] <predicate> | ( <search_condition> ) }

[ { AND | OR } [ NOT ] { <predicate> | ( <search_condition> ) } ]

[ ,...n ]

<predicate> ::=

{ HL7Path { = | < > | ! = | > | > = | < | < = } string_expression

| HL7Path [ NOT ] LIKE string_expression

| HL7Path REGEX string_expression

| HL7Path IS [ NOT ] NULL

| HL7Path [ NOT ] IN ( string_list )

}

Query Examples:

No where clause. Useful for exporting data

select MSH-7 as 'Date/Time', MSH-9.*, PID-3, PID-5.*, PV1-2

Find any illegal SSNs

select MSH-7, MSH-9.*, PID-3, PID-5.*, PID-19 as SSN

        where PID-19!='' AND PID-19 NOT REGEX '^\d{3}-\d{2}-\d{4}$'

Female birthdays < 1953 and male birthday less 1945

select MSH-7, MSH-9.*, PID-3, PID-5.*, PID-7, PID-8

    where(PID-8='F' AND PID-7<='1953')

      OR (PID-8='M' AND PID-7<='1945');

Female patients that contain the last name 'SMITH'

select MSH-7 as 'Date/Time', MSH-9.*, PID-3, PID-5.*, PID-7

    where PID-8='F' and PID-5 LIKE '%SMITH%';

Present, but Invalid SSN 'SMITH'

select MSH-7, MSH-9.*, PID-3, PID-5.*, PID-19 as SSN

    where PID-19!=''

      AND PID-19 NOT REGEX '^\d{3}-\d{2}-\d{4}$';

No PV1 Segment

select MSH-7, MSH-9.*, PID-3, PID-5.*, PID-19 as SSN

    where PV1-1 is null;

--region: Find Patient by original identifier, or by their new patient identifier

select MSH-7, MSH-9.*, PID-3, PID-5.*

    where PID-3 IN ('800292182','800191207');

HL7 PATHS

An HL7 Path is defined as:

SEG[SEG_NUM]-F[REPEAT_NUM].C.S

where:

• SEG – a 3 character segment name. For example PID.

• [SEG_NUM] – SEG_NUM is an integer value where SEG_NUM>0. If [SEG_NUM] is omitted, SEG[1] is assumed. For example OBR[2] means the second OBR segment, whereas OBR[1] and OBR are equivalent.

• F – is an integer value where F>0. For example PID-3, means the 3^rd field in the PID segment.

• [REPEAT_NUM]– is the Field Repeat number where REPEAT_NUM>0 or REPEAT_NUM='*'. If REPEAT_NUM='*' a concatenation of all repeating fields is returned. If [REPEAT_NUM] is omitted, F[1] is assumed. For example PID-3[2] means the second repeating field of PID-3, and PID-3[*] means PID-3[1] + PID-3[2] + PID-3[3]...PID-3[N].

• C – is the Component number where C>0 or C='*'. If C='*', a concatenation of all components is returned. If C is omitted, C=1 is assumed. For example, PID-3.2 means the second component of PID-3, and PID-3.* means PID-3.1 +PID-3.2+PID-3.3...PID-3.N.

• S – is the Subcomponent number where S>0 or S='*'. If S='*', a concatenation of all subcompents is returned. If S is omitted, S=1 is assumed. For example, PID-3.2.3 means the third subcomponent of the component of PID-3, and PID-3.2.* means PID-3.2.1+PID-3.2.2+PID-3.2.3...PID-3.2.N.

HL7 Path Examples:

For the following Message:

MSH|^~\&|SPC|M||M|20040503223716||ORM^O01|176201653|P|2.2|

PID|1||0000307656^^^M&FEE&&FIE&&FOO&&FUM^MR~0000858462^^^P&FOO&BAR^MR

OBR|1||3844834|2035^NM HEPATOBILIARY DUCT^MRD|||200405030939||

OBR|2||44834|21493^RADIATION THERAPY PLANNING|||200406020800||

PID-3[*] = '0000307656^^^M&FEE&&FIE&&FOO&&FUM^MR~0000858462^^^P&FOO&BAR^MR'

PID[1]-3[2].1 = PID-3[2].1 = PID-3[2] '0000858462'

PID[1]-3[2].* = PID-3[2].* = '0000858462^^^P&FOO&BAR^MR'

PID[1]-3[2].4.1 = PID-3[2].4 = 'P'

PID-3[2].4.* = 'P&FOO&BAR'

PID-3[2].4.2 = 'FOO'

PID[1]-3.1.1 = PID-3.1.1 = PID-3 = '0000307656'

OBR[1]-4.* = OBR-4.* = '2035^NM HEPATOBILIARY DUCT^MRD'

OBR[2]-4.* = '21493^RADIATION THERAPY PLANNING'

where PID-19!='' AND PID-19 NOT REGEX '^\d{3}-\d{2}-\d{4}$';
 

This release contains:

- Support for opening a file by double-clicking on it within Windows Explorer.

- Support for exporting Statistics to Excel, PDF file, and printer. See figure below.

- Miscellaneous bug fixes, and performance improvements.

Fixed a particularly difficult to track down crash bug. Thank you Joel for reporting the usage scenario that caused this crash.

Our apologies. The previous release broke the Custom Code feature.

Added a new search dialog which allows you to search on a combination of fields. For example, in the picture below we are finding all messages in the date range of April 2004-May 2004 that have a diagnostic code starting with the word "CHEST".

Also, in addition to a few minor bug fixes, we have added the ability to copy the contents of the data grid in the Statistics View to the clipboard. The data is formatted in such a way that it can be easily pasted into Excel for display.

This is a minor bug fix release.

The only feature added is the ability to search for fields greater than, or less than a specific value in the Search dialog (Ctrl-F). This is mostly useful for search filtering date ranges. It was requested by one of our customers.

This is primarily a bug fix release, though we did manage to add a few small features.

1) There are now options to delete ACK messages with Acknowledgement Code equal to 'AE', and/or 'AR'. In the previous release, it was only possible to remove ACKs with Acknowledgement Code equal to 'AA'.

2) After HL7Spy performs a clean-up, a summary dialog is now displayed which details the operations performed during the clean-up. There is also an option to display any messages that had parsing errors during the clean-up procedure.

3) A SQL History feature was added to the SQL Loader tool which stores the 40 most recent SQL queries. This time-saving feature keeps your favorite queries at your fingertips.

Also in this release are improvements to the Custom Code and Sql Loader tools.

The major feature in this release is the SqlLoader tool. This tool enables users to extract HL7 messages directly from an Oracle, Sql Server, MySql, ODBC, or OLEDB compliant database. It is super fast, and super slick.

To use this feature connect to a compliant database using the Connect button, write a SQL query and let the tool do the rest. SqlLoader will automatically detect which column contains the message data and load them into HL7Spy where all the great features such as searching, sorting, and statistics generation are available; just as if you opened the messages from a file.

 We have added the ability to edit the descriptions in the Watch List. This feature is handy for documenting non-standard fields and Z-segment.

The field descriptions now show up as part of the column names when exporting messages to Excel (see below), and also in the generation of Watch List reports.

Although HL7Spy is extremely fast, users are opening files with a million messages, or more. Depending on the format the messages are stored, this can take a minute even in HL7Spy.

Often one only want to load up messages from a portion of a file. This new feature allows you to do this.

A new dialog is automatically displayed if a file you have selected for loading is greater than 50 MB. It allows you to control the range within the file in which HL7Spy will scan for messages, and it also allows you to limit the number of messages that are read.

Hopefully you find the user interface intuitive to use. The 'Summary' text describes what portion of the file will be scanned, so you should be able to figure out how to use it by changing the knobs and watching the summary text change. 

• Standard 0x0bMSH...0x1c0x1d
• Single message in the file with no framing bytes
• Carriage return separated messages
• Carriage return with Line Feed separated messages
• Japanese framing (omits the 0x0b character at the beginning of the message)
• EGate log file format
• Pyxis log file format
• STC Datagate formats (several flavors)
• IDX ConnectR log format

If you have a format that you would like us to support, please email us a sample file and we will do our best to include support for it.

HL7Spy will do its best to choose the correct file format. If it fails to select the correct one, you can always force it to a know preset, or you can configure your own message framing. To change the message framing options, select from the toolbar the Message Framing button.

The following dialog will be presented. Which allows you to change the message framing characters, change the character encoding of the message, and replace any unwanted strings.

Support for automatically detecting and loading HL7 messages from EGate log files.

Includes other minor bug fixes and improvements as well as some infrastructure changes to support some new features coming out at the end of the month. One of which is the ability to create and manage message definition profiles. This feature will enable users to edit the field, component, subcomponent, and table definitions for different flavors of HL7.

In this release we added a Hex Editor to toolset provided in HL7Spy. The editor is capable of editing very large files while utilizing very little memory and resources. It supports copy, paste, search, and go-to offset functionality.

Additonally, we are displaying non-printable characters in red to make them easier to locate. The end-of-segment byte is displayed as ¶ and other non printable characters are displayed as ▒.

We have also fine-tuned some of the keyboard shortcuts. A listing of the shortcuts can be found under the Help/Keyboard Shortcuts menu item. They are repeated below:

Ctrl+Tab	Switch Focus between the Editor and Segment Editor
Ctrl+P	Open message property window for the selected message
Ctrl+E	Show Segment Editor view
Ctrl+Shfit+M	Merge Message tabs
Ctrl+W	Show the Watch List view
Ctrl+Shift+S	Sort Message tab
Ctrl+Shift+R	Generate watch list report
F1	Show HL7 Standard view

Ctrl+Left	Move to previous non-null field
Ctrl+Right	Move to next non-null field
Alt+Right	Go to next message
Alt+Left	Go to previous message
Ctrl+Home	Go to first message
Ctrl+End	Go to last message
Ctrl+(0-9)	Move to a specific field number. Eg Ctrl+19 move to field 19 in the current segment.
Ctrl+G	Go to specific message index.

Ctrl+I+S	Insert Segment
Ctrl+I+F	Insert Field
Ctrl+I+R	Insert Repeat
Ctrl+I+C	Insert Component
Ctrl+I+T	Insert Subcomponent
Ctrl+Delete	Delete current Selection
Ctrl+Z	Undo edit
Ctrl+Y	Redo edit
Ctrl+Shift+C	Copy message
Ctrl+Shift+V	Paste message

Ctrl+F	Show Find Message dialog
F3	Search Next
Ctrl+Shift+F	Quick Find Message text
Ctrl+Shift+Down	Quick Find Next
Ctrl+Shift+Up	Quick Find Previous
Ctrl+Shift+H	Highlight specific field within the message

Ctrl+N	Open new message tab
Ctrl+O	Open File
Ctrl+X	Close File (clicking on the message tab with the middle mouse button will do the same)
Ctrl+Shift+X	Close all files
Ctrl+S	Save File
Ctrl+H	Open file with the hex editor

Ctrl+Up	Go to previous segment
Ctrl+Down	Go to next segment
Right	Go to next field
Left	Go to previous field
Ctrl+T	Toggle “Show Empty Components” and “Show Empty Fields”

Ctrl+Shift+W	Add current selection to Watch List
Ctrl+Space	Show statistics for selected component
Ctrl+Shift+Space	Show statistics for selected field
Ctrl+Mouse	Hovering over a field with the Ctrl key pressed will display a tooltip with the field description

The theme of this release is keyboard usability.

Most buttons and menu items now have equivalent shortcuts. Please hover over the buttons to bring up the tooltip and the identification of the particular shortcut.

There is better integration between the Message Editor, and the Segment Editor. Ctrl+Tab will toggle between the two editors. There are also shortcuts for displaying/hiding fields and components with empty values, and for moving between segments in the Segment Editor.

Additionally, a hex viewer was added to display the binary data of a message, to for instance, find invalid characters within a message. Thanks goes to Albert from Cleveland Clinic for suggesting this feature.

This is a first cut of this feature. In the next release we will be adding the ability to scroll to other messages within the hex viewer, as well as the ability to edit the raw message data.

This release contains some improvements to the Watch List feature. The changes are as follows:

• Support for multiple Watch Lists
• Ability to reorder the items in a Watch List using drag-and-drop
• Ability to export Watch List items for all messages into a CSV, or Microsoft Excel file.

New Features
- Improved scrolling speed when traversing a large message collection

- Better handling of messages in the Statistics View for those that lack a Trigger Event (MSH-9.2), but contain an Event Type Code (EVN-1).

- Double-clicking on a Component in the Statistics Page will bring up Component Statistics for the Message type that is active. In the figure below, if the segment pointed at by (1) is clicked, statistics for all PID fields will be displayed on the right. If however, the segment under the message type ADT^A08 is selected (3), then PID field statistics will be displayed, but only for messages of type ADT^A08.

Double-clicking on (2) with (3) selected will display the field statistic for PID-8 for all ADT^A08 messages (see image below). The behavior of this dialog is described below:

1) Find all messages with PID-8 that matches the selected value in the grid below. In this case, a new document tab will be openned with all ADT^A08 messages that have PID-8='M'. 

2) Find the next message with PID-8 equal to the selected value in the grid. In this case, the editor will move to the next ADT^A08 message with PID-8 = 'M'.

3) Find the previous message with PID-8 equal to the selected value in the grid. In this case, the editor will move to the previous ADT^A08 message with PID-8 = 'M'.

4) Clicking here will take you to the previous ADT^A08 message that has a null PID-8. (Right-clicking will bring up a context menu with more options)

5) Clicking here will take you to the next ADT^A08 message that has a null PID-8. (Right-clicking will bring up a context menu with more options)

6) Double-clicking will take you to the next ADT^A08 message with PID-8='M'. (Right-clicking will bring up a context menu with more options)

Bug Fixes:
2214:  Searching for specific fields that are null does not work propery
2215: HL7Spy User Interface flashes and is slow to update when resized
2219: Please switch Cntrl-double-click to bring up Field Statistics, and double-click to bring up Component Statistics in the Message Editor and Statistics Pages