Since founding MiserWare became my full time job in 2007, our software has undergone a number of iterations. From its original incarnation as a Linux-only, server-only, TCP-communicating software stack to the latest version of Granola Enterprise, a hybrid web-app (meaning native applications that work in conjuction with a web service) available on PCs, laptops, and servers for Linux, Windows, and VMware, the architecture of our power management software has been tweaked, reconfigured, and completely rewritten more than once. In this post, I’m going to describe our evolution and some of the technology we’ve used along the way.

TL;DR: Lessons learned

• Use a high-level language for native application development unless you absolutely cannot.
• Use the same language for your clients and your backend as much as possible.
• Get your logging straightened out early on.
• Try to encourage widespread deployment of the clients by individuals.
• Rewrite only as dictated by your requirements.
• Don’t be afraid to rewrite if you really need to.

ServerMiser ES: The early days

Our company (and thus the first version of our software) was spawned under very specific circumstances: our research group had been approached by a major international financial organization with a serious problem in datacenter power. They were in the process of putting together a huge datacenter restructuring and consolidation program with a large number of vendors of wildly varying sizes. After performing a paid proof of concept of our technology, they asked us if we would be interested in commercializing it. Long story short, we agreed, and we set about figuring out how to integrate into the web of products in their program.

This framing of the problem, along with our backgrounds, led to some design decisions. The software would be multi-tiered client-server over TCP; the agents would be small and lightweight; though they ultimately wanted more, the majority of their servers were Linux-based, so we would focus on that. Using a subset of C++ that included classes and almost nothing else, we created this software: ServerMiser ES was born. Despite generally abstaining from the more advanced features of C++, we used boost::asio for our networking.

MicroMiser, and the birth of Granola

We knew early on that testing was going to be very important in supporting system software on numerous different environments, so about six months after beginning development, we carved out the core power management technology and created a standalone agent that we called MicroMiser. We ran a little promotion and got a few hundred users, uncovering a bunch of problems along the way.

A few months later, we had hired 3 new developers, for a total of 5. Still cranking on ServerMiser ES, we decided it was time to look into Windows support. After the success of MicroMiser for Linux, we thought that would be a good first step: create a Windows version of MicroMiser, along with a clever little GUI. I broke off to work on this project, while everyone else continued work on ServerMiser ES.

Since our existing codebase was written in C++, I decided to start there; after some refactoring, much of the core code was reusable, though the underlying OS hooks were significantly different, and significantly different among different versions of Windows. Around that time, I had also been reading some books about modern C++, and began integrating features using templating and exceptions. We also began using more of the boost libraries, including smart pointers and functors.

Hollis, who would later become our resident designer, helped us design a fancy little GUI. The GUI code was written first using Windows GDI code, but later was refactored to use GDI+, since it did things like support PNG and transparency. You know, the little stuff. After releasing it, our userbase of MicroMiser grew by a factor of 10, despite still requiring account signup to download. With this release we also included a tool for collecting logfiles and customer feedback easily, and they made a huge difference in the amount and quality of feedback we received while also cutting down on our support touches.

At this point, we knew we wanted to open it up to a larger audience, and based on the advice of a marketing guru who was helping us out, we also knew we wanted to do some rebranding to test some different marking language. Thus, with very little changes to the codebase, Granola was born. The one new piece was a GUI for Linux, created using Python with Glade and GTK+. We created a new website for Granola and removed the account requirement for download. In the 100 days following the release, we had over 100,000 downloads.

Granola matures, and now we need to talk to a website

After the initial explosion of Granola, we were jumping to try and keep people excited. Our green message resonated well with individuals who wanted to do their part for the environment, and we wanted to increase the visibility of that message and of the visibility of the community surrounding the software. In other words, we needed a way to talk to our website.

After looking at the C/C++ libraries for doing this, I decided it would be more flexible and interesting to embed a Python interpreter in our software, having it and the Python standard library do the heavy lifting. It was quite the journey, and I learned a lot along the way, but ultimately it did exactly what we needed.

Meanwhile, we developed a rudimentary API on the backend to support the new communication pathway. Because our site was already written in PHP, and because our needs were simple, we eschewed a web framework and instead crafted our API by hand. End result: Granola clients that could talk to our website and send savings data that would aggregate on a user’s dashboard. We began selling this as Granola Enterprise.

Granola Enterprise and the end of ServerMiser ES

Shortly after creating the first version of Granola Enterprise and as a result of some excellent media coverage we received, we began to get inbound sales requests. After some quick initial traction, we re-evaluated our strategy and came to the conclusion that our new model – client software + web app hybrid – was the new way to go (another story for another day!). Based on this, we wanted to unite the clans and provide the functionality of ServerMiser ES in our web-based dashboard. This minimally included the ability to group machines to aggregate savings and policy information, and to schedule different power management policies by group.

On the web site, we began writing what ultimately became the Granola Dash, creating an API-backed one-page web application, still using PHP for the API and jQuery to handle the AJAXiness. We also defined a protocol for communicating the power management policies that dictated how the clients would act.

On the client side, we needed a more flexible way to handle power management in enterprise settings, so we broke the power management aspect of the software away from the communication bits and called it Granola Manager. We then created a new service, Granola Connect, to handle communication to the website. Like the GUI, it was a C++-with-embedded-Python hybrid. Most of the code was reused between the two, with the only real new code (and only platform-specific code) being the daemonization/service routines and system information gathering.

Around that time, we also began adding VMware support, which looks a lot like the Linux support with some platform-specific data gathering code. Once we were done with that, we no longer had ServerMiser ES, and from the ashes Granola Datacenter was born.

The Unification of Granola Enterprise

Once we had all of that functionality working, we began looking at ways that we could enable what we consider our core directive: making power management as easy and smart as possible. We realized the missing link at enterprise scale was historical and aggregate data. This was reinforced by feedback we were getting from our recently hired VP Sales (who in turn got it from customers). We decided that it was time to revamp the way the core software worked.

I had also been writing an increasingly large amount of Python code, and the embedded Python in the clients was growing. I reckoned it was time to ground-up rewrite the monitoring and communication aspects of our software, this time in pure Python. Let me say, on a personal note, that this was one of the better decisions I’ve made. The resulting code is much cleaner, more readable, more maintainable, more testable, and much much MUCH shorter. The new Python clients are compiled as native executables so that the users don’t need a Python installation. They are also packaged as native packages: MSIs, RPMs, and DEBs. The core power management, still written in C++, is still cordoned off in its own executable.

We designed some data structures to hold the historical data we thought was relevant, and made sure that they would translate well between our clients (now in Python) and our dashboard (in Javascript). As a result of these structures, we created Sanka, a Python queue runner for our backend with a file-based datastore. With this functionality now in the queue runner, the PHP API became not much more than routing and database helper functions. We also added a huge amount of reporting and analytics, allowing us to get a much better handle on our funnel.

With the new historical data in place, we also took an opportunity to merge Granola Enterprise and Granola Datacenter, now a feature called Power Tuning. The clients are all exactly the same, and the license management takes place on our website. This is ultimately how we arrived at our “Measure -> Take action -> Keep saving” mantra. The newly crafted Granola Connect clients enable us to offer the Energy Footprint information for free, after which a user can upgrade their licenses as their need dictates. We also removed the login capability from Granola Personal, which means that our userbase is better segmented.

Conclusion

It’s been a long journey to get where we are. Our software has come a long way, from addressing a very specific problem in the datacenter to being a full-enterprise-scale hybrid web application applicable to laptops, PCs, and servers. We’ve learned a lot along the way, and I’m sure there’s still quite a lot to learn.

In summary, here are my lessons for writing a cross-platform hybrid web app, or really any software:

• Use a high-level language for native application development unless you absolutely cannot. Packaging and building interpreted applications is totally a thing, so don’t use that as an excuse. I suggest Python!
• Use the same language for your clients and your backend as much as possible. Obviously, the scope of this advice is limited, and I’m not sure we’ll start seeing a huge proliferation of Javascript-only native applications, but I could be wrong. Having Python in both the clients and the back end has made my life a WHOLE lot easier.
• Get your logging straightened out early on. Both the clients and the backend need to have really good logging, and there should be really good ways of getting ahold of the logs. Wrap that up in a feedback mechanism as well, and you can gather suggestions and bugs in one fell swoop.
• Try to encourage widespread deployment of the clients by individuals. Testing at the scope of hundreds of thousands of differently-configured systems is a great way to find weird bugs that only arise in certain environments.
• Rewrite only as dictated by your requirements. Though our software has undergone a good bit of churn, I feel like we have only scrapped and rewritten as has been required to support new functionality we wanted. Rewriting ground-up is very expensive in terms of time, and should be avoided as much as possible, particularly when you are in the process of creating both clients and a backend.
• Don’t be afraid to rewrite if you really need to. As I said above, one of the better decisions I’ve made at this company was to rewrite a major chunk of our code. Rewriting is generally bad, but sometimes it’s good.

_____
Joseph

You should consider signing up for a free Granola Enterprise account and establishing your energy footprint.