Archive for the ‘development’ category

Remembering Breakfast

In the past several months I’ve been involved with a rather intense project that involved working on a lot of different parts. Often while I was going through code reviews and analyzing feature requests, something would come up that I could swear had been done previously. Unfortunately, I couldn’t remember if it was directly related to the application at hand or something else I worked on that might’ve had a similar problem.

Often when an issue would be raised about a specific piece of functionality I would get a sense that this had been addressed in the past and promptly reverted because of customer feedback. The problem was I couldn’t pull up any historical data to refute the ticket and explain how the task is a fool’s errand. Another problem was that the lack of history lead to repeating the same mistakes again.

After this happened to me two or three times, I’d had enough. There needed to be a better way to keep track of what I’ve been working on and why. The goal was to be able to either search for a phrase or perhaps some tags and pull up everything that was relevant. Aside from having the benefit of preventing repetition, it also has the added benefit of easily providing status updates.

When I dove onto the internet in a quest to find the journaling software that would meet my needs, I made one tiny mistake; including “mac OS” in my query. This pulled up several applications for mac that would solve my problem but did leave out several good alternatives. I effectively wanted something that would allow me to write journal entries with tags and perhaps send me a nag near the end of the day, something that doesn’t need to be tied to a single platform.

Although in hindsight I could’ve done a better job in looking for software, I also could’ve done a lot worse. The application I chose to use was Day One which met all of my logging needs, and as an additional benefit; is pleasing to the eye.

After having used Day One for a week I inquired on Twitter what everyone else uses. For some it was social media, others use iDoneThis(which I completely forgot about!), to finally more rudimentary tools like vim. It was an interesting insight into how people keep track of what they have been doing throughout their day.

I could see Social Media as note taking or task tracking system perhaps working if you are involved in more open and transparent environments like an open source project. For myself, I’m working on some pretty secret sauce and can’t really use that system for keeping track of what I’ve been working on. Another red mark for me would be I tend to use Social Media for “entertainment”, so I might end up with some false or misleading data in my search results, so this avenue wouldn’t work for me.

iDoneThis is a tool I’d actually used in the past as an alternative to my teams daily stand ups. To be honest this tool is excellent and I’m a bit of a tool for having forgotten about it while searching for a diary software. They support hashtags, which provides the necessary buckets of where a task was related. It is also a separate tool, so I can keep the Signal-to-Noise ratio quite high. Another cool feature about this application is it integrates right with your mailbox, so you can get a message at the end of every day which asks what you’ve done. I’m not too sure how their email integration works with multiple daily updates, but I do know their web interface supports logging multiple things in a day.

One problem that I had with iDoneThis was that I am simply terrible at email. Actionable or important emails I’ll star, and stuff that I need to get to soon I just leave as unread. Unfortunately I put iDoneThis too low on my list of priorities, so my daily updates would get missed. Aside from some workflow issues though, iDoneThis is an excellent tool that comes with the added benefit of being multi-platform.

The last tool I’ll talk about is vim. Well actually it isn’t, because I’m not a sucker1. Instead I’m going to cover vim’s bigger brother, org-mode. This is also another tool I’ve used in the past and even blogged about using it. Org-mode is a fabulous tool that comes with some nice functionality built on top of plain text files with some special formatting. The list of features in org-mode is almost endless, and I barely scratched the surface back when I was using it. I mainly used it as a task tracker instead of a diary so I can’t say much about it’s diary taking features. Considering the scope of the project though, I would be shocked if diaries and tagging weren’t in there. As for why I don’t use it anymore, the answer is quite simple. I’ve sadly moved away from using emacs as my primary editor. I never truly got good at using emacs effectively enough when working on a large codebase. I often had difficulty navigating through various aspects of the code, so I switched over to Sublime Text which doesn’t have the level of org-mode support as the real McCoy.

In conclusion, if you are ever running into a memory-loss problem and are looking for a technical solution, any of mentioned tools might be worth consideration. Of course, this always depends on your workflow and self-discipline.

  1. I’m not serious when I say shit about vim. It’s just not a tool I use, and it’s always a tiny bit fun to get a vimmer all riled up over superficial things like choice of tool.

Digging into the bowels of Rails

Last week was the first hack days for 2013 at Shopify, and this time I was determined to work on some awesome customer facing features.  One of those was working on what I uncreatively called “Virtual redirects”.  The goal was to come up with a way of allowing any route to resolve without resulting in a 301 Moved Permanently status code.

Before I get too far ahead of myself let me first describe what the problem is.

Currently on Shopify we provide the option to have redirects so that you can ensure that your old permalinks would still work.  For example you could have an about-us on your old site, but on Shopify that’s really not how it works.  Your about us page would be located at /pages/about-us and what you’d do is set up that redirect.  Now when a user would visit instead of getting there they’d simply be given a 301 Moved Permanently to the new URL.  While this works it unfortunately results in a somewhat ugly presentation. Instead what we wanted was to simply be able to render the contents of /pages/about-us while giving the facade that the client was viewing about-us.

The reality is that this isn’t absolutely straight-forward and required a bit of digging around in the Action Pack source code.  I had found a few useful posts on various blogs and a few Stack Overflow articles but there was a major caveat; the callbacks wouldn’t fire!  Considering all the state that is required for a client facing Shopify page to render, we definitely needed to get those callbacks firing.

In the typical fashion, I fired bundle open actionpack and started digging through the source code.  I started from the furthest point, ActionController::Base and worked my way down.  Going through the various includes I came across something that sounded a little promising, AbstractController::Callbacks.  While figuring out how to get there now was obvious, at first it required digging through a bunch of the ActionPack code to figure how exactly callbacks were added to a controller.  Once I knew where the callbacks were being added; more specifically what they were being added to, I was able to figure out the exact function that was being called.

Unfortunately this was only a single part of the equation.  I also needed to figure out what this arbitrary URL would resolve to and that required passing the user specified route into our Rails router.  At least in the version of Rails I was using, I could simply get the intended controller back by doing the following.

def resolve_route(path)
  target = Rails.application.routes.recognize_path(path)
  # Making an assumption that all of our controllers are simple like "ThingController"
  # This probably won't work this namespaced controllers like "Internal::ThingController"
  target[:controller] = "#{target[:controller].capitalize}Controller"

With the above we now had the necessary information to fire off our controller’s action.

# inside a controller so we have access to all our necessary context
# `target` is the result from our previous #resolve_path function call
def fire_action(target)
  controller = target[:controller]
  controller.request = request
  controller.response = response
  controller.params = target[:params]
  # Process is the action that actually fires
  # all your expected callbacks/filters
  # Should probably be rendered in the parent
  {:text => controller.response.body, :content_type => 'text/html', :layout => false}

Now you might be wondering why I am passing back raw render parameters like that.  This was more of  a time constraint than anything and there was some wonky stuff going on in Shopify’s customer facing controllers that would result in weird stuff happening.  I am quite confident that it would be possible to pull off simply calling a controllers action via process though I didn’t get the opportunity to verify this.

Security Concerns:

While this does ensure that various callbacks and filters are fired I’m still not confident that this is an exceptionally safe way to implement calling other Controller actions from within a controller.  If one isn’t careful I’m pretty sure that a malicious user could come up with some crafty way of viewing information they really shouldn’t.  I’d have to test this out more, and because of that I wouldn’t consider adding this to anything in the near future.  If you had a single controller that was providing the routing you could simply instantiate that controller instead.  At least you know the user wouldn’t be doing any crazy stuff and possibly getting access to something their eyes shouldn’t be seeing.

Performance Concerns:

This felt exceptionally noticeable while working with the application in development.  This essentially requires rebuilding the world all over again in order to establish the proper context within the desired controller, just to fire off that action.  This is simply another reason why I wouldn’t consider using in it’s current implementation in a production system, the overall cost is way too high to justify a somewhat toy feature.


If I were to come back to this problem, what I’d want to do is come up with a safe and secure way of allowing users to specify their own routes in a way that the Rails router could figure out the appropriate action and simply do all the setup on its own.  I feel that this would be the cleanest and most precise way to provide such a feature, the unfortunate thing is this would effectively be giving users access to your applications routes files.  It’s already hard enough to trust our users to play nicely in our applications, so adding a feature like this would definitely need some severe kind of sand boxing, otherwise the effects could have some really unfortunate consequences.

The tricky path to writing data to an NFC tag

I’ve been making some good progress on my brewkit application. One big sticking point was figuring out how to reliably detect and write to an NFC tag. While the tutorials and guides on the official Android site were pretty solid, I was still left confused and not really knowing how the whole detect and write to the tag part worked.

I found it relatively tricky to find good articles about working with NFC. This is when it dawned on me that CyanogenMod had support for changing your phones settings based on various “profiles”. What you would do is open a profile, tweak it, then when you were happy write it to a chip. What was also awesome is the all the CyanogenMod code is publicly available. So instead of flailing around trying to figure out how the NFC cycle works, just read some source instead!

A quick google search later I found the exact application I was looking for and was off to the races.

There’s really only a few things that need to be done to get yourself rolling with writing to a tag.

1. Have the NFC permissions
2. Get the proper intent filters set up
3. Handle incoming intents fired when an NFC tag is detected
4. Write to the NFC tag

1. Have NFC Permissions

This one is pretty simple to get going, all you need to do is add a uses-permission flag to your AndroidManifest file

  <uses-permission android:name="android.permission.NFC" />

2. Get the Proper Intent Filters Set Up

While we can add the intent filters in our manifest file, we might not always want to handle them. I noticed in the CM app that they didn’t have these handlers set up in their manifest. Though, we want to handle when any tag is detected so that we can overwrite them and this isn’t appropriate except when that activity is visibly active. So what we’ll need to do is enable those intent filters during runtime.Instead, they only do it when the activity has been created. Also, since I’m using fragments, having my app always handle

public void onResume(){

public void onPause(){

private PendingIntent getPendingIntent() {
  return PendingIntent.getActivity(this, 0, new Intent(this, getClass()).addFlags(Intent.FLAG_ACTIVITY_SINGLE_TOP), 0);

private void enableNfcWrite(){
  NfcAdapter adapter = NfcAdapter.getDefaultAdapter(this);
  IntentFilter tagDetected = new IntentFilter(NfcAdapter.ACTION_TAG_DISCOVERED);
  IntentFilter[] writeTagFilters = new IntentFilter[] { tagDetected };
  adapter.enableForgroundDispatch(this, getPendingIntent(), writeTagFilters, null);

private void disableNfcWrite(){
  NfcAdapter adapter = NfcAdapter.getDefaultAdapter(this);

So now when a tag is detected our application will handle that tag.

3. Handle incoming intents fired when an NFC tag is detected

Now that we are ready to handle when a Tag is detected we just need to add the appropriate callback, that being the onNewIntent method. Here we simply check to see if the intent was for one where we actually discovered a new NFC tag. If it was we handle it.

public void onNewIntent(Intent intent) {
  if(NfcAdapter.ACTION_TAG_DISCOVERED.equals(intent.getAction())) {
    Tag discoveredTag = (Tag) intent.getParcelableExtra(NfcAdapter.EXTRA_TAG);

Wow, that was exceptionally simple! That’s all we need to do to grab the Tag.

4. Write to the NFC tag

Now for the grand finale! Let’s get some data written on that NFC tag. To keep things brief I am not going to deal with all the exceptions that would normally have to be dealt with

private void writeSomeStuffToTag(Tag tag) {
  Ndef ndefTag = Ndef.get(tag);
  byte[] stringBytes = "Hello World".getByes();
  NdefRecord dataToWrite = NdefRecord.createMime('fancy/niblits', stringBytes);
  ndefTag.writeNdefMessage(new NdefMessage(dataToWrite));

And there we have it! We’ve written some data to the record. If you’d like to confirm that some data is there get your hands on one of the generic NFC apps available for Android. My personal favourite is NFC TagWriter by NXP. Fire it up scan the tag you just wrote to and take a look. You should see the data you wrote to it (at least the mime-type) being reported from the chip. Pretty awesome stuff!


Keeping Your CTags up to Date

I switched over to Sublime Text recently and became a huge fan of it’s CTags plugin. This has become a vital tool since at work most of the stuff I’m working on is in a huge codebase and it’s tricky to investigate the moving bits without them.

Unfortunately, I don’t seem to have the CTags extension running properly and had to go in and re-run the ctags tool again so I can once again easily re-navigate the codebase. Clearly this calls for some automation!

Of course I’ll pull out trusty old cron since I don’t need anything more complicated than that.

Pretty much all I did was hack together a quick little script then put the following into my crontab:

30 9 * * 1,2,3,4,5 ~/.scripts/buildctags.rb

Then the script, which is pretty ghetto but gets the job done


places_to_process = ["~/development/ruby/"].map{|dir| File.expand_path(dir)}
places_to_process.each do |directory|
  entries = Dir.entries(".").reject{|i| i.chars.first == "."}
  entries.each do |entry|
    Dir.chdir(directory + "/" + entry)
    system("ctags -R -f .tags")

Easily create a simple row/column layout using each_slice in ruby/rails

I was looking for an easy way to go through an ActiveRelation so that I could lay the data out in a sort of grid-like manner.  Initially I went some hacky way that involved checking modulos and inserting divs.  It was a gross abomination and I won’t go into any further details.

Instead, I went around looking for something that was a far nicer solution.  In my search I came across this lovely little function `each_slice` which allows me to say “break this array up into chunks this big and iterate through them in little chunks”. This was exactly what I wanted and it worked out amazingly!

<!-- myview.html.erb -- assuming I'm using Twitter Bootstrap -->
<% @products.each_slice(3) do |row| %>
  <div class="row"> <!-- Each row of my Grid -->
  <% row.each do |product| %>
    <!-- Each item in my grids row -->
    <div class="offset1 span3"><%= product.title %></div>
  <% end %>
<% end %>

Now, I’m not sure if it decomposes the AR into an array (I’m hoping it doesn’t), but it’s a lovely way to present the data that is easy to read!

Of Cooks and Programmers

I’ve recently been thinking of what differentiates a good from a bad programmer. Often people have used words like “elegant” and “simple” as ways of describing attributes of a good programmer. Though surely we can come up with a physical analog to draw a better comparison. Personally, I’ve found the analog to be cooking because it’s easy to tell a bad (or junior) cook from a good one.

Previously I was training in the military as a cook, so I’ve taken several courses that taught various skills such as organization, discipline and of course; how to work in a kitchen. Much of the kitchen work heavily revolves around how one treats their workspace. Of course this is important to ensure that the kitchen is safe and that there is no contamination, which can lead to extremely disgruntled clients. While my profession has drastically changed, I still take those lessons from my culinary experience and apply it to my own kitchen.

When it comes to my own kitchen I’ve noticed a pretty big disparity between how my partner and I prepare our meals, that being while she’s really good at creating the end product, the side-effects of her cooking are rather… messy. Anyone can be a good cook, it just takes a bit of discipline and the desire to keep your space as clean as possible. This of course includes things like having a sink full of hot, soapy water so that in between steps you can clean the tools you just used.

So you’re probably wondering what does any of this have to do with programming? Well if we compare how a good and bad programmer works, it’s quite similar to working in a kitchen. We all start with a clean slate and start combining the pieces to create the end product, though the good programmer will continue to return to their code and find ways to improve upon it. This can be anything from modularizing code, to simply taking a piece of code and making it simpler to follow.

A bad programmer on the other hand will leave their solutions because “it works” and they’ll get back to it eventually. The problem with this is your workspace gets more and more cluttered, increasing your chances of having your code contaminate other parts of the codebase and before you know it, your system is sick. Now you’re stuck with a semi-working system that they cannot understand because of all the short-cuts and bad decisions that were made but never taken care of.

Now, I do believe that like a bad cook, a bad programmer can be taught the proper way to build things. Just like trades people need to spend several thousand hours as an apprentice, developers also need to do this. Without the correct mentorship we are never given the guidance we need to grow into a good developer. Well, perhaps that isn’t entirely true, you’ll still have the chance to grow though it will take a bit longer and you may also build up some bad habits.

tmuxing my Way to a Better Future

So it’s been about 3 years since I made the switch over to Macs, and I’ve been pretty happy with it. I have shell that doesn’t suck and all the goodness that comes with a UNIX-like operating system. The one thing that’s always bothered me a bit is the lack of good tiling window management. About 6 months ago I played with xmonad for a bit and really loved it, even to the point where I tried using it in XQuartz on Mac (which turned out to be a terrible experience).

I’ve somewhat managed to get away with it by using emacs for the most of my editing. While not of the same calibre as a TWM like xmonad, it gives me that frameless experience I like. The one disadvantage is the lack of a shell I could use (if there’s something better than eshell, OH PLEASE TELL ME!) so I’ve typically had emacs using 80% of my screen, and an iTerm window using the other 20%… or something. It’s been alright, though I hate having to switch out of one application into another run something then switch back, etc etc. This is where I decided to take the plunge and give tmux a shot.

For those who don’t know about it, tmux stands for “Terminal Multiplexer” and allows you to have multiple terminal sessions running, sort of like Spaces or Virtual Desktops except entirely text based. They also allow you to disconnect from a session while not actually killing the processes, which is useful for things like running something on a server (upgrading stuff, or running a really long query where timeouts may be an issue). The connecting/disconnecting part has been a pretty awesome life-saver for myself where I have a tendency to lose context and kill or close the wrong application which is most often my editor.

Along with the session management there’s the window or pane aspect as well, which I really just use as a tiling manager. It’s not as advanced as xmonad, but I’m spending more time in a single application (iTerm) where it’s drastically easier to switch between my applications.

Of course everything isn’t perfect! I still haven’t figured out how to get stuff from emacs into my clipboard, which kinda sucks though aside from some sort of minor issues like this, I’ve been pretty satisfied.

Please think of the Users: Smarter Task Queueing/Scheduling

I’ve been using a certain Podcast application for a while, and I have it configured in such a way that it will only do updates on wifi and at 9:30 in the morning.  I have a tendency to only be connected to 3G so this leads to some pretty frustrating experiences.  Primarily, I’m getting notifications that updates couldn’t happen then I’m directed to a screen with no way of actioning on these problems.  Alright, so the software isn’t that great… but it’s sadly the best for what I need.

Though, I feel this software could be vastly improved if only the developer took a few steps to gracefully handle things such as connectivity (let’s admit it, lack of connectivity isn’t an exceptional case, it’s an expectation).  Add in configuration such as “Only download over WiFi” well now you’re going to need to elegantly handle how an update is performed.

In my app, STO on the Go I needed to perform daily updates though there’s a good chance that when the scheduled task hits the user may not have an internet connection (or only want to download updates over WiFi).  Awesomely, Android is a really loud system.  It’s yelling things **all** the time about various status changes, one of these being connectivity status changes.  I figured I would tie into this.

Tying into these services is extremely straightforward, first we need to tell our manifest that we want to listen in on some BroadcastIntents:

<receiver android:name=".receiver.ScheduleUpdateReceiver">
<action android:name="" />

So now our <pre>ScheduleUpdateReceiver</pre> will get called each time the state of connectivity changes on the device. Keep in mind, you are going to need to keep the code as lean as possible until you are absolutely sure you need to do some heavy lifting! Nothing is more frustrating than an app that appears to be doing nothing but kills your battery.

Now it’s time for our service to actually do some stuff, this is where we will be asking a few questions.


public void onReceive(Context context, Intent intent) {
	ConnectivityManager manager = (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE);
	TelephonyManager telephonyManager = (TelephonyManager) context.getSystemService(Context.TELEPHONY_SERVICE);
	String doUpdateOnNextWifiConnection = context.getString(R.string.doUpdateOnNextWifiConnection);
	SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(context);

	if(!prefs.getBoolean(context.getString(R.string.performDownloadsInBackground), true)) {
		// User configuration -- No background Updating
		Log.d(DEBUG_TAG, "Updates are not permitted to run. Aborting");

	boolean hasActiveWifiConnection = manager.getNetworkInfo(ConnectivityManager.TYPE_WIFI).isConnected();

	// Check to see if the intent that was fired was created by Android itself
	if(intent.getAction() != null && intent.getAction().equals(ConnectivityManager.CONNECTIVITY_ACTION)) {
		// Did the last check fail due to lack of connectivity?
		boolean performUpdateOnNextWifiConnection = prefs.getBoolean(doUpdateOnNextWifiConnection, false);
		if(performUpdateOnNextWifiConnection && hasActiveWifiConnection) {
			// Alright, our update has run, time to reset our state so we don't attempt to download
			// another update next time the device gets an internet connection.
			prefs.edit().putBoolean(doUpdateOnNextWifiConnection, false).commit();
		} else {  
			Log.d(DEBUG_TAG, "Update for connection to wireless access point will be ignored");
	} else {
		// This is our scheduled update
		boolean canDownloadWithMobileData = prefs.getBoolean(context.getString(R.string.usenetworktype), true);
		boolean hasMobileDataConnection = (telephonyManager.getDataState() == TelephonyManager.DATA_CONNECTED);
		if((canDownloadWithMobileData && hasMobileDataConnection) || hasActiveWifiConnection) {
		} else {
			// So, we either don't have an internet connection or the user wasn't connected to WiFi when the scheduled
			// update was supposed to happen.
			// I chose to store this information in the shared preferences because it's pretty easy to do.  This could get
			// a little bit out of hand if you have more complex rules than what I have here.
			Log.d(DEBUG_TAG, "Cannot do a download over data and we don't have a wifi connection. On the next wifi connection, we'll do the download then");
			prefs.edit().putBoolean(doUpdateOnNextWifiConnection, true).commit();

So now we are able to handle the sort of special cases and deal with them almost immediately after a user has acquired a usable connection to the internet.

Deadline Met! STO on the Go is available on the Android Market

I gave myself a personal goal around mid to late November; to finish an STO Android app with the dataset that Phil Casgrain had given me access to.  The app is now available on the Android market as STO on the Go though it’s almost impossible to find using the search.

I’m selling the app for $1.00 though there is a good chance it will be getting an increase in the next version, which will have at least maps and favourites features.

Diving further into Clojure

As I mentioned earlier I decided to start playing around with Clojure on my Windows box. This was an excellent way to get a quick introduction to the language, although there were aspects of the Windows platform that made going any further than toying around somewhat pointless. Although, this could simply be my bias towards Unix style systems.

When I decided to set up clojure on my Ubuntu installation, it felt like there was a huge wall in my way that made absolutely no sense. This was really because I had made a few mistakes when installing my editor of choice (emacs) from apt. That being, I had installed various emacs plugins such as swank and slime which caused some really frustrating problems to arise when running swank-clojure. I found in the end, the easiest thing to do is get a full install of emacs going, without any extra bells and whistles; that’s what the ELPA is for, and it does a damn good job at it.

Once my emacs was back to a good state, I was able to get the clojure-jack-in function to work the way I was expecting it to. Finally I was able to get hacking! On the windows box, I was (and still am) doing a text adventure game from the Land of Lisp book. I’ve found this to be an excellent way of stepping into the language, especially since the book assumes you are using common lisp. This has lead to a fair amount of research into finding out clojure equivalents to CL macros. Meanwhile, on my Ubuntu machine I’m working on a project that I’m hoping will make one part of my life a bit easier.

I’ve found that when it comes to doing tasks, such as groceries, it can often be a pain. My partner and I use our phones for entering our grocery list and each time it’s excruciating. These keyboards are annoying to use, our Nexus ones have those poorly placed heat sensitive buttons that your thumbs always manage to hit, and the program we are using doesn’t learn very well. So I’ve decided that the first step in making this better would be to have an application that makes at least entering the data easier. I’ve decided that I’m going to implement this tool using Clojure, because I’ve found I’m loving the way you do things in the language (it feels more mathematical). I also feel that since I’m extremely green with the language that I’ll be required to keep things simple (I still cannot get [& args] working for whatever reason!).

In an attempt to keep things simple, I’m avoiding the database… well that’s a lie. I’m avoiding the Relational database. I’ve always found that when I use a tool like Rails I tend to over-complicate my models for whatever stupid reason and it just leads to me abandoning the project.  I’m also interested in playing with document stores (in my case MongoDB) since I love to use hashes/JSON.  It just so happens that there is a library called congomongo which is pretty easy to setup and start playing with MongoDB. Once I’m done getting this part of the application working, my next plan is to play with the Noir microframework (similar to Sinatra or Flask) to implement my web app.  If I decide to continue working on this, it probably won’t scale all that well, but at least it’s simple enough to not overwhelm me at first.