Maintaining Custom Forms for a Clinic

Published: December 13, 2021

Updated: December 13, 2021

As I mentioned in my December 2021 Update, for most of 2021 I have been working at a medical clinic maintaining and updating the custom forms used by staff to enter patient data into Centricity, an electronic medical record (EMR) program. The majority of this work falls in two main categories: custom forms programmed in a proprietary scripting language, and HTML and Javascript based forms that run on the EMR's API in a captured internal browser instance. A small amount of additional, more miscellaneous work is also needed on occasion.

The custom scripting language, MEL, is an artifact from the mid 1990s designed by GE. It lacks many features, is pretty obtuse and archaic, and can only be added to EMR forms via a visual form editing software. The visual editing software, and the tools it includes for working with function libraries, seem not to be designed with a programmer in mind at all - I can only assume they never expected anyone to do any serious customization with it, but rather expected it to be used for minor edits and customizations to the vendor supplied forms. (You can't even open two functions in the same library at the same time and have them side by side!) I inherited a base of code in this, with parts being 10 or more years old (at least one form was dated from 2005), some with minor or no updates since. A lot of the existing code was extremely dense and difficult to understand. Much of my work on these forms has been to update this code and reduce bugs, mostly by making things more clear through comments, renaming variables, and trimming dead code. And documenting dependencies, since the compiler - such as it is - won't warn you if you took out something you were actually using.

I've done as much as I can to introduce reusability into this code base, but it's difficult because the clinic has multiple specialties, each of which has their own custom set of forms, and a change on one type of form requires changing the same spot on about a dozen forms. It is possible to reference function libraries for this, but due to the way this works one has to be extremely careful changing any of them. All the forms in the EMR will reference the same set of libraries, so you can't change how a function call works on a new version of a form without potentially breaking every old version of the form that may still be active in patient updates (which sometimes can linger for a month or more until a physician finishes and finalizes them!)

The previous IT director had a plan to update all of these forms to Javascript/HTML forms eventually. I'm uncertain at this point if we'll be staying with Centricity long enough for this to happen (or for that matter, if Centricity will be retired before then), but I make gradual progress on it among my other duties.

The other part of my responsibility at this job is the Javascript forms. The code I inherited here was mostly written by the two prior developers in my position. This code was in separate chunks, each of which had its own dependencies and its own versions of libraries in use. Mostly it was using AngularJS in a range of versions from about 1.3 through 1.5. Some of it was full of one- or two-character long variable names in seas of nested for loops; some of it was written in a way completely interlinked with code in an MEL form such that the MEL form was creating strings for the Javascript to evaluate as code and vice versa. It was...fun.

My initial priority here was getting this base of code up to date for a software update that the clinic was going to do on the EMR software, scheduled within a couple months of my joining the company. The biggest compatibility hurdle here was that much of this code relied on making ActiveX calls in Internet Explorer (which the older version of the EMR used as an internal browser) in order to interface with the EMR, but the new version has an API that lives at window.external and uses Chrome as the internal browser (approximately version 57). So no more ActiveX.

Some of the more recent forms actually used the new API already and required minimal work to function correctly in the new version of the EMR, but there were two major ones that I had to substantially rewrite. In the process I moved them both into one single page application (SPA) in order to share EMR interface and processing code and updated them to use AngularJS 1.8. (Side note: this also involved some aggravating hurdles with URL rewriting and a JBoss server of indeterminate version...)

These two forms were written by two different people and had very different approaches and challenges. One of them kind of used the API for its calls, but didn't work correctly with the new version, and was difficult to understand (written by the developer who liked to use the aforementioned one- to two-character variable names...). In a way it was the more challenging of the two, since I got it more or less working without rewriting it all initially, or even deeply understanding it, but then had to come back and update and rewrite more of it more times to resolve issues in the future. The other one was the one that lived half in a MEL script, and that one I had to largely reconstruct from the ground up in order to get it working.

As I did this I discovered that it was very difficult to test any Javascript code for a couple of reasons, on top of the hurdles that any developer in any environment will have to deal with. (Although I am sure other environments have their own unique challenges as well!) The biggest ones were, first, if code was loaded outside of the EMR environment, the page often would not even fully load as errors would throw from trying to call API functions that didn't exist; the second, the fact that in the EMR environment where the functions exist, the developer has no access to browser dev tools whatsoever (and no way to get console logs!).

The second I partially solved by including a script that would print any console output to a div on the bottom of the SPA page, thanks to some help from Stackoverflow. The first issue I was able to get in check by adding a lot of try-catch statements. I'm sure that's an overall upgrade anyway, though.

This left the fact that to do live testing with real data, I would have to load the EMR in the test environment between every update (and hadn't yet figured out how to cache-bust with the tools we had available, so also had to manually clear the cache every time.) The easiest solution to this was to build a unit test suite and supply it with mocked data, so I introduced unit tests to the build as well. This enabled me to iterate much more quickly on the code as I worked on it, and verify my logic, parsing, and other code handling aspects. I set this up about halfway through migrating the second of the two big forms I mentioned above, and it sped up my process substantially.

At this point the SPA was working in the new EMR, the upgrade happened, I hunted and bug fixed a couple issues we hadn't noticed beforehand in some of the minor forms, and I was looking at the future of the SPA and being able to upgrade it more easily. At this point I knew there were plans to adapt a number of other forms into this format in the future and wanted to be sure it would be as easy as possible to add more content. And, AngularJS is scheduled for end-of-life on December 31, 2021. Not to mention, it lacked a lot of the nice features that began to be introduced with Angular 2. So I started an upgrade path for getting things working in the current version of Angular (then 12).

Initially, following recommended procedures, I converted all of the code to use ES6+ syntax, especially modules and import syntax. This was an interesting hurdle when I discovered that the browser inside the EMR doesn't support the type="module" syntax for importing Javascript. (This was also one of my several "later rewrites" of the form I mentioned above as complicated...) I had to also introduce webpack, I think it was about this point, in order to build it into a syntax that the EMR's browser could understand. With no mentor and no guidance, this was obnoxiously difficult; there are so many versions of Javascript out there, and so many libraries that handle imports in various ways, to someone only 3 or 4 months into JS it felt like being lost in the wilderness. Every tutorial seemed to have different syntax and it was rare to find one that gave enough information for a step by itself; many assumed you were using one environment and already familiar with it, or another.

But imagine the triumph at getting it built and loading it properly in the test environment for the first time! :D

Of course then the next thing I did was add another layer of complication to this process by putting everything in Typescript, since that's the recommended language to use for Angular. I actually like Typescript a lot better than raw Javascript, though. The IDE tools seem better, and the strong, static typing makes me feel more secure, and makes things easier to reason about.

That's another thing I've been doing consistently whenever I handle these files: I've been doing my best to make things easier to reason about. Making objects shallower; defining regular interfaces; taking data structures that were, for whatever reason, arrays of key: value pairs (I mean, [key, value], not [{key: value}]) nested inside other arrays and turning them into objects with defined interfaces; extracting chunks of code inside deep nested loops into their own functions, and so on. Break things up into pieces so each part can be tested and understood on its own, instead of having to figure out a 700 line function by itself.

I hope whoever handles this after me appreciates it, but realistically I get the feeling we will have a new EMR and have to start over with a different interface for customization (if any) by then.

As I migrated everything into Typescript, I was also rewriting it for Angular 12. I briefly ran a hybrid application, which had some weird issues that were hard to get rid of, and always gave errors on build. Eventually everything was migrated, I still had weird errors, and I couldn't get Angular's CLI to build it... so I created a new project and migrated my files one by one. Or more accurately, a group of related files at a time, since one by one was not really possible.

Finally, error free builds in Angular 12! I have some regrets over losing the git history, but at the same time, there was no git tracking these files before I got there, so it was only my own commit history I was leaving behind (and it still lives in a copy of the old repo, for now; I don't think it will stick around much longer, though, as it's not needed.)

In the process of getting this all working I got frustrated with the time it took to clear the cache and reload the EMR with every change, no matter how minor. Back into JBoss I went. I finally located a cache filter that we already had installed - so no worries about introducing a new dependency - and found the documentation for it to learn how to set cache times. Using no-cache didn't work, but setting a max age of 0 did; so between that and Angular's cache-busting content hashes, I was able to set the SPA up to never cache index.html, while caching content-hashed .js files, so that users would not have to make excessive downloads of things that hadn't changed. Finally again - navigating off and back onto a page in the EMR would show my most recent production build, no more making users wait for bug fixes.

Once this was all set up and working properly, I moved on to converting the remaining HTML based forms into routes in the Angular SPA. Again, these were still written in various versions of AngularJS, and while I used the existing code in one window as a guide, I ended up largely rewriting most of them except the most simple.

After getting all these forms rewritten and moved, and catching up the backlog of tasks that had been left by previous developers, I kind of ran out of things to do. I have been working on adapting some of the MEL forms into HTML, but it has been hard to get user feedback so I am a bit stuck where I am at. I guess the good thing about that is, I don't have anything more important to do when someone does come up with a new request.

Hopefully I get a new real project soon.

This post is tagged: