Cohort Builder status in screenshots

I thought it would be nice to post some screenshots of the current state of the cohort builder.

After defining a cohort, the default patient list view is displayed:

Cohort builder: List View.

The user then has the option of selecting other views, namely the Gender/Age chart view, or Concept chart view. Selecting the Gender/Age link, the following is shown:

Gender/Age chart view.

This shows the cohort patients separated according to gender and age. Clicking a pie slice filters the cohort to include only the corresponding patients:

Gender/Age chart view after filtering by adult female patients.

That's about it for this view. The Concept view offers some more functionalities...

Concept chart view

By default, the CD4 Count frequency distribution is shown. The concept values are divided into ranges, and the chart shows the number of patients in each range of values. Hovering on a bar shows the correspondent range and number of patients in that range:

Hovering frequency distribution bar.

The bars are clickable and, just as the Age/Gender chart, clicking filters the data shown. In this case, clicking the [277-314] range, produces the following:

After clicking range.

For numeric concepts, there is also the option to view a Patients vs Values chart. This plots the values for every patient in the cohort and can be useful for detecting critical patients. When available, the critical values for the concept are also shown:

Patients vs Values.

Both types of charts (Frequency and Values) allow zooming into specific areas, by clicking the reddish band on the X axis and defining a zoom "window":

Zooming.

Resulting in:

After zooming in.

Zooming out can only be "simulated" by going back to the previous view, using the browser Back button. A different solution needs to be studied for this

And that's what's been keeping me busy in a glance... Some other remarks:

• Coded concepts are plotted in a frequency pie chart:

• Datetime concepts are plotted in bar charts, but the Patients vs Values chart is not available (not applicable)

• Browser back button works, to step back after zooming in, even though one never actually leaves the cohort builder page. Though I love AJAX, the fact that it breaks the back button is possibly one of its most annoying aspects. Luckily there are several solutions to it, and after some research and tweaking, this one did the trick.

And what I guess will keep me busy for some time:

• Buttons or links to go back to previous view or go back to original cohort.
  
• The option of using the most recent, the least recent or all of the observations is still under development.
  
• There is a code refactoring in progress, to aggregate values directly on the database instead of in Java in order to improve performance. This implies massive refactoring in DWR services, the JSP cohort portlet and in the Flash charting library as well, so I imagine this will take quite some time...
  

I'll post some more technical details once the code is a bit more stable.

Cohort Charting demo

Looks like there's finally something nice to show. A demo page will be made available to show some possible interactions between cohorts, and concept charting. Basically, one is able to pick a concept and have it graphed for a set of patients.
It's not yet a stable OpenMRS component, but still a very promising one.
Drop by #openmrs this Wednesday around 3pm EST if you'd like to know more...

Work with Open Flash Charts

After a longer-than-expected struggle to find a decent charting library, it looks like we will building our own, using Open Flash Charts as a starting point.

OFC already comes with a nice set of features... Plus, and most importantly, it's open-source and free, meaning we can do whatever we want with it (or at least whatever GPL allows us to do).

Although a great part of the library is organized in classes, a significant part of OFC is just bundled on the .FLA file, namely code that is shared by all types of charts available. This has some disadvantages:

1. Encourages "spaghetti code", stuffing everything with no specific organization into the Flash file instead of in a separate ActionScript file.
2. Doesn't make concurrent development easy: having a diff done on an Actionscript file is readable, while doing it on a .FLA file isn't.
3. Does not allow development inside Eclipse. See below why coding Actionscript is much easier in Eclipse than in the Flash authoring environment itself.

Having said this, I have already successfully translated all the code in the .FLA into a separate class file, maintaining all existing functionalities, plus the ability to have Javascript links associated to each chart value. Such a simple task took a lot of my time, until I found that when embedding flash objects in HTML, you need to set:

<param name="allowScriptAccess" value="always" />

instead of 

<param name="allowScriptAccess" value="sameDomain" />

if you want to test your files locally. Otherwise SWF movies will not be able to communicate with Javascript through getURL() or ExternalInterface.call(). Unfortunately it took a long time to find this on the web:P
I also changed the way in wich parameters are passed to the SWF movie. Instead of loading them from an external file as it's done in OFC, e.g. using:

<param name="movie" value="open-flash-chart.swf?data=mydata.txt" />

they are now passed as flashVars themselves, e.g.:

<param name="movie" value="open-flash-chart.swf?values=2,5,1&x_labels=Mon,Tues,Wed" />

This allows us to fetch information dynamically, for example with AJAX, and pass it directly to the movie without the need to create an external file.


As for the question "Why is it better to code Actionscript in Eclipse IDE?", once you try the FDT Eclipse plugin you'll understand... In short, it gives you all the functionalities you would have if you were programming in Java. Actually, there's no advantage in using it if you don't go object-oriented, but otherwise it gives you auto-completion, error highlighting, quickfixes, among others... I really recommend this, or at least some other tool that does the same job (if you know of some open-source alternative let me know).
You'll end up using the Flash authoring environment just for publishing your files.

Next steps:

• decide on the best way to pass parameters to Flash from Javascript (this is already possible but it can probably be improved)
• have OpenMRS building the code to include the open flash chart
• create pie charts! (not available in OFC)
• whatever else comes...

Playing around with charts...

Just as I suspected, maintaining a blog is not so easy... But after a long time, here are some more updates.

The interface and first charts to implement are now outlined. There will be a default List display of the cohort patients. It will be possible to switch between different visualizations, to start with these will focus on a Gender & Age patient distribution, and some kind of yet-to-be-defined encounter distribution for the patients in the cohort.

I have been working on building an abstraction for provide a simple way to include charts in any OpenMRS page. The original idea was to create a chart tag that will take the chart parameters and return the formatted chart code to include directly on the JSP page:

For example:
<openmrs:showchart width=" height="200" type="genderAndAge" patients="listOfCommaSeparatedPatientIds" />

would be replaced at runtime with the chart itself, keeping things transparent and to the programmer and general enough to be reused in different places.

As simple as this may seem, I ran into some problems, namely the fact that openmrs: tags seem to be replaced before page load, so something like:


$('cohort_contents').innerHTML = '<openmrs:showchart patients="'+listOfCommaSeparatedIds+'">';

would be loaded by the browser as:

$('cohort_contents').innerHTML = '<object><chart_contents_here></object>';

making it impossible to have run-time updates to the chart.


So the alternative (still "pending approval") is to have a function on DWRPatientSetSetvice (or possibly in another specific chart service), that will be invoked with all the parameters necessary, plus the ID for the DIV that will hold the chart, and return the Javascript code that will render the chart. This has been successfully tested with the gender and age charts, with good results (screenshots to come) and seems to be the way to go.



Another thing that's been troubling me is which chart library to use. I have tested both Dojo and FusionCharts.
Even though Dojo charting widget is FOSS and uses SVG, it's quite rudimentary in my opinion, and lacks in looks (poor chart design) and functionalities (no pie charts, no hyperlinks on series, no values over columns, ...). It works fine with the existing functionalities though.

As for FusionCharts, it looked like the perfect thing... while it worked. I played around with it for some days with no problems. Then is stopped rendering charts inside OpenMRS pages. Some days later worked fine again, then stopped rendering again, with no apparent reason. I've spent endless hours around this, clearing every possible cached files, reverting source files, redeploying, even reinstalled Tomcat, but no luck. It just works randomly (other more rational suggestions are welcome:P). The demo for the commercial version seems to have a more stable behavior, but the problem remains. If the cause of this isn't found, the FusionCharts alternative will have to be abandoned...

There is another worthwhile option: Open Flash Charts. I heard about them before from Sashikanth (a fellow SoCcer), but since they use PHP libraries I thought it would be hard to implement. However, when I looked into the source code today, I realized it should be fairly easy to translate it to Javascript, and this is worth a try.

So the current TODO list is:

• Define which service to use to deliver charts to JSP pages, move the existing code into it, clean it up and evolve from it.
• Check out Open Flash Charts, possibly translate into Javascript and try it (ping Sash about his progress with this)
• Improve performance of Gender & Age chart, and start working on the other relevant charts.

About the Cohort Definition Tool

As promised, some details about the project...

My original application was for the Clinical Data Visualization Tool, but a slight re-allocation happened and I'll be working with the Cohort Definition Tool.

First of all, the defining the term Cohort is important. At least for me it was, since I wasn't familiar with it. According to the trustful Wikipedia:

"In statistics and demography, a cohort is a group of subjects — most often humans from a given population — defined by experiencing an event (typically birth) in a particular time span. (...)"

Even though this is a very general definition, it's quite applicable to this context. Medically speaking, and particularly in OpenMRS, a cohort is a set of patients with common characteristics. These can be very diverse, from gender, age, types of drugs taken, etc... After defining a cohort, one should be able to act on that set of patients as appropriate, e.g. scheduling an encounter for all male subjects with CD4 count<350.

Darius, who will be mentoring me for the Summer has made very significant progress with the Cohort Builder, as seen on OpenMRS demo website:

A user can already search cohorts according to many variables, conveniently organized in categories and available through the tabbed interface. Cohorts can also be combined with AND and OR operators, or by defining a more complex expressions. The searches can be saved for later use and are stored with cached results (which may not be always accurate, but guarantee a fast UI response to user queries and a good estimate of the number of patients).

As complete as the described tool may be, there's always room for improvement, as with any piece of software. Currently the patients' names are listed together with their age and sex. My work will focus on graphical cohort visualization alternatives. One of the main goals is to have this data organized in one or several plots, customizable by the user. The exact way in which this will be done is still to be decided, but the adopted solution should provide a clean, fast and useful way of viewing cohort data.

More on this to come...

Hi everyone!

Summer of Code 2007 is on! I will be using this space to post the ups and downs of this beachless Summer.

"OpenMRS formed in 2004 as a open source medical record system framework for developing countries — a tide which rises all ships. OpenMRS is a multi-institution, nonprofit collaborative led by Regenstrief Institute, Inc. , a world-renowned leader in medical informatics research, and Partners In Health, a Boston-based philanthropic organization with a focus on improving the lives of underprivileged people worldwide through health care service and advocacy."

From openmrs.org

I heard about OpenMRS a few months ago, and got interested from the moment I checked out the nature of their work. It's a great thing to be able to use what I've learned to help those who need the most, and that's the main reason why I chose to get involved in an OpenMRS project.

Just a little something about myself... This is not my first SoC, I also participated last year, implementing some SVG filters support for Inkscape.org, which was a really great project to work on. SoC is an amazing initiative, especially for those not-so-familiar with Open Source development (as I was, until last year) for so many reasons... I especially enjoyed the amazing personal experience I got out of it, the community bonding that is created among the participants, all the new things you learn about the mentoring organization and the fact that I helped to build something that will be put in use by so many people (check out the Gaussian Blur applications on Inkscape 0.45).

For all this year's OpenMRS SoCcers (students, interns or whatever you'd like to call them:)), good luck for your projects, I'm sure you'll have a great time!

This introductory post explains the WHY, stay tuned for the WHAT and HOW of my project ;)