Latest Posts from csaunders


Mental Health and Toxic Communities

From Model View Culture about Toxic OS Communities.

We need more stories of people leaving toxic communities so that people know it’s ok to prioritize their mental health over their community. We need stories of successfully switching jobs, of abandoning positions of power to protect our mental health or get a better work-life balance.


Curious Cat

curious_cat

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License.
Permissions beyond the scope of this license may be available at http://sndrs.ca.

 


Landing in a Storm

Landing in a Storm

I wouldn’t say this is the best photography I’ve done and it was super hard to get the focus right on the spot, but I wanted to share a shot from a pretty cool light show I got to see that made me decide to stay up a little bit longer.

I need to spend some more time using my equipment so I can get some better in the moment shots like this in the future.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License.
Permissions beyond the scope of this license may be available at http://sndrs.ca.

Learning to Deploy Servers with Ansible

provisions

I’ve been wanting to learn how to use a configuration management tool for a long time. In the past I’ve attempted to learn Chef though I found myself getting lost amidst the domain language. Trying to use these tools made me feel extremely stupid simply because I was having a hard enough time grasping the concepts of the system.

A couple of weeks ago during our Dev Show and Tells, Nate Smith showed off Ansible. Ansible a configuration management tool but it’s goals are to be easier to use and understand. It’s written in python and uses YAML files for defining your provisioning tasks, called Playbooks in Ansible.

Before we can start using a Playbook we need to do a tiny bit of setup first. You’ll need to define your Server Inventory which is required before Ansible can go out and provision your servers for you.

# hosts
[webservers]
web1.example.com

[databases]
db2.example.com

Now let’s verify that our host configuration is correct:

/Users/csaunders/development/ansible [csaunders@outerhaven] [23:09]
> ansible all -i hosts -m ping
web1.example.com | success >> {
    "changed": false,
    "ping": "pong"
}

db1.example.com | success >> {
    "changed": false,
    "ping": "pong"
}

Now that we know that we are able to connect to our servers, let’s build a Playbook. A Playbook is simply a definition as to what kinds of tasks you’ll be running to setup the server. The simplest playbook you could have would be one where you install apache2 on a bunch of servers:

---
# Here we define the hosts we are going to run them on
- hosts: all
  user: root
  tasks:
    - name: Install Apache with php
      apt: pkg=apache2 state=present
    - name: Install mod PHP on Apache
      apt: pkg=libapache2-mod-php5 state=present
    - name: Install PHP
      apt: pkg=php5 state=present
    - name: Install the PHP MySQL Bindings
      apt: pkg=php5-mysql state=present
    - name: Restart Apache2
      service: name=apache2 state=restarted

What I really like about this is it’s very easy to understand what is going on. We have a little description that describes what the task is and the command that will be run. The name of the task will also be echoed into the terminal when you run the playbook so you have an idea of what task to look at if something goes wrong. And while you might be thinking that the above is pretty verbose, Ansible also provides the tools such that you can run the same command on a list of things. So you can shorten the above example into a single command.

- hosts: all
  user: root
  tasks:
    - name: Install Apache2 and the necessary PHP modules
      apt: pkg={{ item }} state=present
      with_items:
        - apache2
        - libapache2-mod-php5
        - php5
        - php5-mysql
    - name: Restart Apache2
      service: name=apache2 state=restarted

Often we want to be able to re-use a bunch of the code we write because many different kinds of servers use many of the same things. This is where we can define Roles which we can assign to various hosts or groups of hosts. A Role consists of a number of things though the major components are the tasks to run, templates to use and handlers that fire after a task.

Now with a few roles defined you can create Playbooks that leverage them:

---
- hosts: webservers
  user: ansible
  sudo: yes

  roles:
    - common
    - mysql
    - apache2

All the tasks contained in each role will be executed, and if all goes well you will have a server that’s ready to serve up some MySQL powered sites!

So far I have to say that I’m quite happy with Ansible. I’ve been meaning to migrate my systems around because they were somewhat out of date and I wanted to bring some costs down. The idea of doing it all manually was a massive deterrent from doing it whatsoever simply because of all the work that’s involved. While it might have taken me more time to get what I needed to do in Ansible; I know that if I ever need to do something like upgrade WordPress, it just requires changing a configuration value and running a command. It also makes it a lot easier for me to do things like change providers if I ever wanted. Learning to use this tool has also laid a base from which I can build upon. The process for creating a configuration to prepare a server for a Go or Ruby app should be far simpler! If you are looking to level up your dev ops knowledge, then I’d definitely consider looking into Ansible as a way to do that.


The Problem with shopify_app’s Session usage

An issue was recently opened on the Shopify App project because of a change in what Rails allowed you to set inside a session. This caused the project to break for people looking to play with the API simply because they couldn’t set their session up correctly.

I’ve rarely used the gem simply because its generators are very noisy, but after talking to a friend about this I became extremely aware of what the problem was. The following is the offending code:

  sess = ShopifyAPI::Session.new(
                            params[:shop],
                            response['credentials']['token'] 
                          )
  session[:shopify] = sess        
  flash[:notice] = "Logged in"
  redirect_to return_address

Now why is this a problem? By default, Rails uses cookies for storing sessions and cookies are sent off to your client. Thankfully the cookies are run through an HMAC so they cannot be tampered with, but the data is still raw and available for anyone to read.

Reading the Session

This can easily be shown by grabbing the contents of our cookie from chrome:

broken cookies yo

There are a few things that need to be massaged before we can read the data, that primarily being the trailing –ae63169593d0e2e68a72977324f5791d6d31b8b8 which I believe is the HMAC. Also the %3D needs to be converted back into a = for us to work with the data correctly. You can use a utility like Rack::Utils#unescape to take care of this if you don’t want to do it by hand. The contents of the cookie are simply Base64 encoded, so let’s decode the message and see what’s there:

require 'base64'

msg = "BAh7C0kiD3Nlc3Npb25faWQGOgZFVEkiJTZmZWVlNzlhOTA4YzBlOGQ3YzYwM2U3ZmY4OWRjMGFkBjsAVEkiDnJldHVybl90bwY7AEZJIgYvBjsAVEkiEF9jc3JmX3Rva2VuBjsARkkiMW1MVlJITnFoNjJ4UllScDA2VEpPNkVoQWpLSkZoaVF2eFlnczIyMDhrV0E9BjsARkkiE29tbmlhdXRoLnN0YXRlBjsAVEkiNWU4NTJmZTM1YzZjZWYyYmY2NWJjYjIzY2RjMDllOWRkODkyZjUwMzE4YTNlMWNiNgY7AEZJIgxzaG9waWZ5BjsARm86GFNob3BpZnlBUEk6OlNlc3Npb24HOglAdXJsSSIbanVzdG1vcHMubXlzaG9waWZ5LmNvbQY7AFQ6C0B0b2tlbkkiJWI4YmMxNjQyMmUyODFhODM3MDhhN2NlNmIxYzRhZWZiBjsAVEkiCmZsYXNoBjsAVG86JUFjdGlvbkRpc3BhdGNoOjpGbGFzaDo6Rmxhc2hIYXNoCToKQHVzZWRvOghTZXQGOgpAaGFzaHsGOgtub3RpY2VUOgxAY2xvc2VkRjoNQGZsYXNoZXN7BjsNSSIOTG9nZ2VkIGluBjsAVDoJQG5vdzA="

raw_contents = Base64.strict_decode64(msg)

raw contents

Woah… so that’s looking pretty hairy and by just looking over it visually we can see some sensitive data in there. But let’s make things even easier on ourselves. We can use the Marshal module to actually convert that data into objects that are super easy to read and use.

# require the libraries so we can unmarshal the data
require 'action_dispatch'
require 'shopify_api'

session = Marshal.load(raw_contents)
#> {
    "session_id"=>"6feee79a908c0e8d7c603e7ff89dc0ad",
    "return_to"=>"/",
    "_csrf_token"=>"mLVRHNqh62xRYRp06TJO6EhAjKJFhiQvxYgs2208kWA=",
    "omniauth.state"=>"e852fe35c6cef2bf65bcb23cdc09e9dd892f50318a3e1cb6",
    "shopify"=>
      #,
    "flash"=>
      #,
        @closed=false,
        @flashes={:notice=>"Logged in"},
        @now=nil>
   }
shopify = session['shopify']
puts "Domain: #{shopify.url}, Access Token: #{shopify.token}"
> Domain: justmops.myshopify.com, Access Token: b8bc16422e281a83708a7ce6b1c4aefb

Now it’s plain as day! We were storing raw Shopify session objects in the cookies that we are sending back to our clients and we were able to decode them. Depending on your configuration this could mean that malicious third parties could hijack your access token and do some really bad things to a merchants shop.

How do I avoid this?

The rails security guidelines suggest against storing any kind of sensitive data in the session and this style of vulnerability is the #2 in the OWASP Top 10 vulnerabilities of 2013. The default way that it was done in shopify_app was definitely something you’d want avoid in a production environment. Unfortunately nothing was put in place to help provide the education needed to ensure people moved away from this practice.

When building an application you want to ensure that sensitive information never goes anywhere you cannot trust. While it is convenient to store the data in the session, the client is about as untrustworthy as you can get. What should be stored in the session is something that you can use to identify the data you need.

How to Fix it

In a patch that has been merged into master, you instead configure a ShopifySessionRepository such that it knows how to store and retrieve sessions from your server. For example, you could simply have your Shop model respond to those two methods and you’d be set. When a session is stored you return the ID for the model such that you can retrieve a ShopifyAPI::Sesssion when you need it at a later date.

A new version of the shopify_app gem was just released, so you can easily upgrade your application by simply performing gem update shopify_app which will upgrade you to version 5.0.0. If you are starting a new app you’ll want to ensure that your Gemfile is using the newest version such that you are able to use the safer session implementation.

gem 'shopify_app', ">= 5.0.0"

If you are using the shopify_app in your project, take a look at your sessions controller to verify that your show action doesn’t have the offending code as mentioned above. If it does, you should remove that code and replace it with a safer way to access the credentials associated with a shop. Consider looking into what was added to shopify_app for one approach to this problem.


Building Games for Fun, Feedback and F…. Learning

Last week I did a presentation at the Ottawa Ruby group. Luckily we were running a live stream of it and I was able to grab a copy of the raw video. You can check it out below:


Using Bresenham’s Line Algorithm for Raycasting

Last week I wrote about my first stab and implementing a proper field of vision for my rogue-like. The implementation left me somewhat disappointed because it didn't really work for what I wanted.

After sitting down with the code and studying it a bunch I realized why the code wasn't working and what would need to be done to improve it. The problem rested in the fact that it simply iterated over these octants but was completely ignorant of any state around in one of the directions. The outer loop had no context as to when we should stop incrementing. The result was being able to see through eastern and western doors.

Another problem I had with this implementation is it didn't give me that really nice viewing region that I've grown to love in rogue-likes. The algorithm was too aggressive and would allow me to see through thin walls and so fourth. It had to be rewritten. One thing I liked about using this suboptimal algorithm (for my purpose) was that it gave me a better understanding of what I was doing and how to solve it.

I went back to the wikipedia article and started implementing various versions of Brasenham's Line Algorithm.

The first version was the Optimization of the algorithm. I went with this one first because I found the code was easier to understand and would help ease me into the simpler/faster ones. After implementing the algorithm one of the flaws in it became somewhat apparent.

hmm something isn't quite right

The first optimization doesn't do anything to establish the direction of where the rays are coming from. The result was that rays would be cast inwards toward the point instead of outwards. I needed them to be cast outwards since that would give me the wonderful fields of view we get when entering rooms or corridors that we see in our rogue-likes.

The second optimization does exactly what I wanted. It determines how to move outward from starting point towards the end of the line. During the line-building process, if I encountered an opaque object, I'd kill the line drawing then and there.

The one thing with the line algorithm is that it just does a line, we are still responsible for telling it where all those lines should be cast. I saw a couple of guides on how to do it but they felt really awkward. It involved using macros to do things a bunch of times which just felt wrong to me. Since we are casting rays from a point in a circle, instead of using octants or whatever, why don't we just us polar co-ordinates?

The nice thing about polar coordinates it they give us a perfect circle around our character and the calculations from polar to cartesian are extremely easy to understand.

oh yeah I need to add the original point to my calculations

After a tiny bit of head scratching because I forgot to add my starting point to the calculations I had my rays!

eugor in action


Casting Rays in Roguelikes

I’ve gotten back into building my rogue-like again. It’s good to be spending a little bit each day on the various parts of it. My main focus recently has been on implementing the fog of war / vision such since I’ve always found those to be the funnest parts of a rogue-like.

Raycasting in DoomRL

Most rogue-likes do a field of vision that has a maximum range and also allows you to into corridors. One of the ways this is implemented is using Ray Casting. The way this works is you choose a point (often the player) and cast rays from their position in all directions. When a ray hits an opaque object the ray stops, otherwise we mark that position on the map as seeable and keep casting the ray. Typically we’ll cast these rays in the various octants because we want our Field of Vision to be around the player.

My initial implementation was going to use Bresenham’s line algorithm but I wasn’t really getting any results and it was sort of tricky to debug (hint, I was getting stuck in loops). I hit up roguebasin to grab one of the Ray Casting algorithms that people had decided to share. One thing that really stood out for me was that it had a very old-school programmer feel to it. Maybe I’m bad at math, but the code was somewhat tricky to understand. I’m also not sure if media wiki nommed up some mathematical symbols because there were some really important mathematical functions missing. I want to really understand what is going on, and this is even more important since I’m implementing the game in Go not C, so I can’t even copypasta this. Understanding what is going on is key or I’d just end up with magic in my codebase and I hate magic.

After a bit of confusion and head-scratching I was able to get my implementation of the algorithm is working alright. There were a few parts in the roguebasin example that I skipped, mainly because I don’t think they were necessary and couldn’t actually understand what was going on. Considering the results I have, I am pretty happy to have avoided implementing that code.

My implementation of raycasting does make a few assumptions:

  1. The raycaster is initialized with the dungeon/maze/map
  2. The maze is what is responsible for fetching the tiles
  3. Tiles provide the information regarding their opacity

The meat of the algorithm is as follows (Note: I’ve reordered the code such that you can read it top to bottom):

func (r *Raycaster) flushOverlay() {
    for x := range r.overlay {
        for y := range r.overlay[x] {
            r.overlay[x][y] = false
        }
    }
}

var OCTANT_CALCULATIONS map[int][]int = map[int][]int{
    0:  {1, 1, 1, 0},
    1:  {1, -1, 1, 0},
    2:  {-1, 1, 1, 0},
    3:  {-1, -1, 1, 0},
    4:  {1, 1, 0, 1},
    5:  {1, -1, 0, 1},
    6:  {-1, 1, 0, 1},
    7:  {-1, -1, 0, 1},
    8:  {1, 0, 1, 0},
    9:  {0, 1, 1, 0},
    10: {-1, 0, 0, 1},
    11: {0, -1, 0, 1},
}

func (r *Raycaster) calculateFieldOfView(x, y, intensity int) {
    for _, values := range OCTANT_CALCULATIONS {
        sx, sy, dx, dy := values[0], values[1], values[2], values[3]
        r.doOctant(x, y, intensity, sx, sy, dx, dy)
    }
}

// Algorithm from Rogue Basin
// http://www.roguebasin.com/index.php?title=Ray-Tracing_Field-Of-View_Demo
func (r *Raycaster) doOctant(x, y, radius, sx, sy, dx, dy int) {
    for i := 0; i != radius; i++ {
        var lastTile *dungeon.Tile
        var lastAdjacentTile *dungeon.Tile
        for j := 0; j != radius; j++ {
            tileX := x + (sx * i)
            tileY := y + (sy * j)
            if tileX >= r.maze.Width || tileX < 0 || tileY >= r.maze.Height || tileY < 0 {
                break
            }
            tile := r.maze.FetchTile(tileX, tileY)

            adjacentTile := r.maze.FetchTile(tileX-(sx*dx), tileY-(sy*dy))
            if lastTile != nil {
                if lastTile.Name != "wall" {
                    r.overlay[tileX][tileY] = true
                } else {
                    if tileX <= 0 {
                        break
                    }

                    tileIsWall := tile != nil && tile.Name == "wall"
                    adjacentTileIsClear := adjacentTile != nil && adjacentTile.Name != "wall"
                    lastAdjacentTileIsClear := lastAdjacentTile != nil && lastAdjacentTile.Name != "wall"

                    if tileIsWall && adjacentTileIsClear && lastAdjacentTileIsClear {
                        r.overlay[tileX][tileY] = true
                    } else {
                        break
                    }
                }
            }
            lastTile = tile
            lastAdjacentTile = adjacentTile
        }
    }
}

// The public interface is the following
func (r *Raycaster) CastRays(x, y, intensity int) {
    r.flushOverlay()
    r.calculateFieldOfView(x, y, intensity)
}

func (r *Raycaster) IsLighting(x, y int) bool {
    return r.overlay[x][y]
}

Using the ray-caster is fairly straight-forward, I’m going to assume that the maze is already initialized:

raycaster := MakeRaycaster(maze)
raycaster := CastRays(5, 10, 10)

The result of using this in my game is the following.

Raycasting in Eugor

There are a few little bugs that need to be fixed, those primarily being able to see through walls and some walls being hidden when looking directly at them.

As always, you can peruse the source on github at your own pleasure.


Reflections on TO Jam 2014

TO Jam Notes

The past weekend I participated in my second game programming competition called TOjam. The goal was to simply make a game in under 48 hours, similar to Ludum Dare though with a local community to build your games with.

Compared to the Toronto Global Game Jam, I'd say things definitely went a lot better for me this time. The big reason was because I was on a team so we were able to distribute work and test iterate over the ideas relatively quickly.

The main idea of the game was to build out a western-style shootout but with princesses as the people in the standoff. The theme changed around a few times though the end product was something we were all super happy with.

Day 1 – Spinning

The first day, Friday, consisted of us mainly trying to figure out how we wanted the mechanics to work in order for the shootout to happen. One thing I thought that might've been interesting was to have a QWOP style interaction with drawing your pistol. This could've ended up being something interesting, though given my lack of experience with Box2d and how janky the experience felt we decided to scrap it.

you-are-the-dead

We also weren't super sure on what kind of art style we wanted to go with. We decided to spend a bit of time trying out a few different art styles, though it was uncomfortable and definitely slowing the designers down.

Day 2 – Locking down the mechanics

When I came in on Saturday we did a brief discussion about what we were looking to do and how to get there. I started working on more prototypes using boxes and such until I had some rough assets to drop in.

Most of the work for me consisted of getting all the collision detection working, controlling the various player states and managing things like the bullets that were being fired. One part of the project that was starting to really hurt was how player state was being managed. Initially I was simply having it driven by button presses which lead to some really weird edge-cases that would cause players to die when they shouldn't and so on.

This was around the time when one of the designers, David Rusak needed to know how we were going to be handling the animations. Initially I was simply going to perform a bunch of transforms on various parts of the sprite (such as rotating the arm) but after talking to some more experienced jammers at the event, decided to go with frame-style animations instead.

The big thing I wanted to avoid was the use of Sprite-sheets. For super detailed things they are awesome, but depending on your frame rate and how you are updating them, they can be really hard to tweak. Another thing that I was worried about was figuring out when certain parts of the animation had finished. I sat down with David and we chatted about an idea I had that would make it super easy for designers to tweak timings that didn't require developer intervention.

I want to say this worked out really well because we were using Löve2D and Lua which has that lovely benefit where code can be data. Writing out the data-structure was extremely straight-forward, and "parsing" it was practically trivial. Another added benefit was we were able to add events that would be fired when a frame of the animation had completed. This meant I was able to move over to a callback pattern for controlling player state.

Once we had the animations setup I took to helping out the designers with some work. They were super busy working on all their art so I took to building out the various aspects of the UI that could still look good but be built out by a developer. This mainly consisted of stuff like the DRAW! screen and instructions. Initially I grabbed some ancient assets from Microsoft to use as the buttons though they made us pretty sad. They were super dated looking for one, and really didn't fit in with anything we had built out.

three-two-one-draw

By the end of day 2 we had made some awesome progress! There were a few missing pieces, but overall it was at the point where we just needed to add the finishing polish.

Day 3 — The Final Day

The Final Day

It was pretty much all about polish during the last day. I spent a bit of time within Audacity to tweak some sound effects to fill in some of the empty parts of our game. Another thing I spent time on was trying to figure out where the loop points of our game music was. I'd say there is a bug in Löve2d when it comes to playing audio, even when you are playing sounds set to loop. The problem is when the music is over there will be dead air before the music starts up again which is quite jarring.

When we started getting closer to the end of the event, I figured it would be a good idea for us to get together and hammer out all the remaining pieces that needed to be worked on in order for us to consider the game done. There were going to be a few missing features, but a big thing for me was something that didn't crash and played well. 7pm was pencils down, so I figured we could call our tojam release time to be an hour before. This would give us enough time to catch any last minute problems and put together the binaries for release.

Post-Jam

I'd have to say this was one of the better hackfests/jams I've participated in. The one thing that really stood out is there was a vision that was easy to realize and a scope that wasn't too crazy. I also feel that we weren't afraid to cut out things that might've ended up in some features that might've been cool but could've compromised the integrity of the project. Finally, I had chosen to use tools I was familiar with which meant that while I was still spending time learning, the slope wasn't as steep as it's been in the past.

If you're interested in playing the game, you can grab a copy for Windows or Mac. I also suggest that you take a look at the other games that were built during the jam as well!


Breakable Toys: Reflecting on the Rats Nest

Rats Nest

It's been a while since I've talked about any of my progress on my rogue-like eugor. Things were coming along pretty well though I've gotten to a point where a bunch of naive decisions are finally starting to bite me.

Thankfully there were a few areas that I picked up on were going to be terrible. I abandoned the idea of using graphs for my map rendering since it added unnecessary complexity for something that didn't need anything like this. I took some ideas from roguebasin and rolled with them. In essence, I have a serialization format that I use for storing basic details.

In general, things are working alright though I am changing a lot of data. I was trying to roll with using interfaces since I have various kinds of structures like Creatures and Characters, though the abstraction somewhat fell apart.

This became apparent one morning while I was trying to make it possible for my character to attack a creature. When I was working with values it was easy, I'd call a function that modified the value and re-assigned it. This approach is pretty gross though, I'm doing all this copying around and it becomes really hard to work with once you start passing these objects around via methods. I've decided that I just need to do more reading on pointers in go and take a look at my game from a distance and start asking myself questions about it.

What sucks right now? Does this abstraction need to exist? Can these objects be represented more generally? Can I justify why this code exists?

It's going to suck to take the stuff I've worked on and throw it out, but it's becoming harder and harder to get an idea of what I'm actually doing. It's time to reflect on what I've built, what I learned while building it and how to make it simpler.