Justin Howard

A Dependency Checklist

Sun, 25 Apr 2021 00:00:00 +0000

Unless you’re working for a large corporation with strict security guidelines, you may not think too much about adding a new third-party dependency to your project. However, even if you’re the sole developer or working on a small team, choosing dependencies should not be taken lightly.

This checklist explains what I consider when considering adding a dependency like a third-party library.

Activity

Is this software actively maintained? How friendly is the author to outside contributions? Are there many outstanding bugs? What was the last time a bug was fixed? If the project has dependencies of its own, is it compatible with the latest versions of those?

I typically will not add a dependency unless I’m confident that issues that I discover can be fixed. I don’t mind fixing it myself, as long as my time isn’t wasted on a pull request that will never be reviewed.

License

For the most part, I look for open-source dependencies with few exceptions. I prefer simple, permissive licenses like MIT, BSD-3, ISC, and Apache 2.0. Depending on the project, GPL may also be acceptable. A project with a missing license is a no-go.

Maturity

For the most part, maturity comes down to how long a project has been used in production. I consider overall age, but also things like performance, configurability, and stability.

A newer project is less likely to stand up to heavy production use where its limits are tested. I prefer not to be a beta tester in these cases.

Documentation

A piece of code can be the most revolutionary, perfect piece of software ever written, but it’s worth very little without good documentation. This comes in the form of guides, API docs, and release notes.

Guides are intended to address the most common use-cases for a piece of software, things like setup and configuration. API docs should dig deep into the full public API for a library and document the details of each class or function. The release notes, or changelog should document all significant changes in each version and specifically call out breaking changes.

Reputation

Who is the maintainer or maintainers of the project? Is it backed by a company, or an individual? If it is an individual, are they known in the community? Can they be trusted? Are they vulnerable to coercion by a malicious party? How likely is it that the project will be abandoned?

Without knowing the reputation of the maintainers, it is very risky to use their code in your application.

Code Quality

When I scan the code, does it have a consistent, readable style? Would I feel comfortable writing a patch for it? Does it have good test coverage? What kind of bugs are in the issue tracker, and do those demonstrate commitment to quality?

Code quality can be an indicator of stability and usability, but it’s also important for maintenance. Code that is a spaghetti mess is likely to become difficult to maintain or have unfixable bugs.

Popularity

How many other users are there of this software? Although popularity isn’t critical, it does make it more likely that the project will continue to be maintained. Especially if the project has a permissive license, a popular project is likely to be maintained even after the current maintainers abandon it.

Popularity also makes it more likely that common bugs will have already been found and fixed.

Transitive Dependencies

Don’t forget to check the dependencies that the project itself has. Transitive dependencies are just dependencies of your dependencies. You depend on these just as much as your direct dependencies, so it’s important to apply the same rigor to them.

Overall Risk

Considering all the other points, what is the risk of introducing this dependency? How easy will it be to replace in my application if necessary? What would be the impact of a bug or security vulnerability if there was one?

Adding third party code to your application is risky, but almost always necessary. Make sure to do your due diligence.

Avoid Ruby Hash Defaults

Thu, 22 Apr 2021 00:00:00 +0000

In Ruby, like most languages the default value for a hash is nil or empty. You can override this behavior with your own default. Here are some reasons to avoid using this feature.

The Basics

You can create a ruby Hash with the literal {} syntax, or with Hash.new. If you use Hash.new, you can specify a default with an argument

Hash.new(0)

Now, any time you access a key that is not present on the hash, it will return your custom default instead of the stored value.

h = Hash.new(0)
h[:a] # => 0
h # => {}

Already, we see some potential traps. Here’s a example:

h = Hash.new(0)
h[:a] = 1 unless h[:a]
h # => {}

Here, the author makes the mistake of assuming that the default of h is nil, but it’s actually 0, so we can’t check for presence with if/unless like we usually can. Okay, we can probably handle that and just make sure to use key? when we want to check key presence, so let’s look at some more complex and dangerous examples.

With Mutable Objects

Probably the most common use of hash defaults is to make the default another hash or array.

h = Hash.new([])
h[:a] << 'foo' # => ['foo']
h # => {}
h[:b] # => ['foo']

Wait, what happened? Where did 'foo' go, and why is it in :b? Well you probably guessed it. When we mutate with <<, we actually mutated the default array. Not only did we not add the :a key, but we also changed the value we get when we request an unset key.

We can solve this problem by using a block to create a new array whenever we request a default. This solves a problem and creates new ones.

With Dynamic Default

We can pass a block (Ruby’s closures) to Hash.new that will get called every time we need the default value. Let’s see how that works.

h = Hash.new { [] }
h[:a] << 'foo' # => ['foo']
h # => {}
h[:b] # => []

Okay, so that didn’t fix it. We’re no longer mutating the default array, but our changes still aren’t sticking. Let’s look at the << line and break it down into its components.

# Equivalent of h[:a] << 'foo'
# First get the default value, so our block is called and returns []
# and we store it in a
a = h[:a]
# Now we append 'foo' to our new array
a << 'foo'

You can see that our array with 'foo' in it never gets assigned back to h. Let’s try to solve that problem.

h = Hash.new { |hash, key| hash[key] = [] }
h[:a] << 'foo' # => ['foo']
h # => { a: ['foo'] }

Nice! We solved all our problems. Or did we? There are still a couple more pitfalls because we now assign hash keys whenever they are accessed.

Access as Assignment

Let’s look at how you might use our solution so far in your application. We’ll see how it can cause issues.

class RequestHeader
  def initialize
    @fields = Hash.new { |hash, key| hash[key] = {} }
  end

  def [](field)
    @fields[field]
  end

  def content_type=(value)
    @fields[:content_type] = value
  end

  def add_cookie(cookie)
    @fields[:cookie] = [] unless @fields.key?(:cookie)
    @fields[:cookie] << cookie
  end
end

We created a class to store request headers, and added a helper method to add cookies individually. Did you spot the error? Let’s look closer.

header = RequestHeader.new
header.add_cookie('cookie')
header[:cookie] # => ['cookie']

Looks fine so far, let’s look at how this breaks.

header = RequestHeader.new
header[:cookie] # => {}

Oops, we expected an array, but it returned us a hash. This is a contrived example, but a very similar bug caused a major issue in production for an app that I worked on. Let’s look at one more trap before wrapping up.

h = Hash.new { |hash, key| hash[key] = [] }
h[:a] ||= ['foo'] # => []
h # => { a: [] }

I Avoid Hash Defaults

Did you catch all these mistakes? If you did, good job, you’re a very knowledgeable Ruby developer. However, I’d prefer to just avoid these issues altogether. Imagine if you initialize a hash in one file, and pass it somewhere else in the application. It would be a nightmare to anticipate or track down this type of bug.

In general, I now completely avoid overriding Hash defaults, and I suggest you carefully consider your own uses of them.

Why I Love Code Comments (Now)

Wed, 14 Apr 2021 00:00:00 +0000

Should I add comments to my code? It turns out this is a controversial topic that has much more nuance than you’d think at first glance. Here are the various approaches I’ve used and why I now write a lot of comments.

A Beginner Approach

As a junior developer I wrote code without a good understanding of how it would be maintained or how it would stand up to continual changes. Then as my experience grew, that code went from an unreadable mess to organized and (for the most part) clear. At this point I rarely used code comments and I didn’t see the need for them.

As my skills improved, my team was able to understand how my code worked. This was a big improvement, but it’s not nearly enough.

Code as Documentation

As I became an intermediate developer, I came across some blog posts that described “self-documenting” code. The concept is that if you write clear enough code, and name your variables and functions well, comments are unnecessary. They often claim that if you need comments, your code needs to be fixed.

Here is an example in Ruby

def user_is_valid?(user)
  return false if user.full_name.nil?
  return false unless user.email.nil? ^ user.phone.nil?
  true
end

The line with email and phone is not readable at first glance since we’re abusing the ^ operator to be XOR (one or the other but not both). We could add a comment, or we could just extract a method to document that.

def user_is_valid?(user)
  return false if user.full_name.nil?
  return false unless user_has_email_xor_phone?(user)
  true
end

def user_has_email_xor_phone?(user)
  user.email.nil? ^ user.phone.nil?
end

This is much more readable. I started following this approach, fanatically refactoring all my functions into self-documenting bits. Today I take a slightly more pragmatic approach, but I still use similar strategies.

However, even this is not enough. A reader can easily understand that we’re validating that a user has an email XOR phone, but why? Why can’t our user have both?

Communicating Intent

The next step of documentation is communicating intent. Intent expresses not only how your code works, but why it is designed that way. All applications are designed to work a specific way, but unless you document intent, that design can be lost.

Let’s say the product manager wants to add support for both email and phone on user accounts. As a developer of this application, I would encounter the validation rule above. I would wonder whether it’s safe to remove this. Is anything downstream depending on this rule? Will the front-end UI behave correctly?

Let’s add clarifying intent comments to our example above.

# The given user has an email or phone but not both
#
# A user must have at least one way to contact them. That can be either email
# or phone. However, users cannot have both an email and phone because that
# would break the NotificationSender module which currently does not support
# users with both contact methods. If we fix NotificationSender to support both,
# this validation rule can be removed.
def user_has_email_xor_phone?(user)
  user.email.nil? ^ user.phone.nil?
end

We now know why this validation rule was created. Later if we want to remove this rule, we have better information and can approach change with the full history in mind.

Much of the code I write now has comments like this. Especially as application logic becomes more complex, intent documentation is invaluable.

Start Thinking About Connection Timeouts

Sun, 22 Nov 2020 00:00:00 +0000

When an application makes an HTTP request to another service, the application surrenders control of its performance to that external service. If it takes 10 seconds to respond, the application waits for 10 seconds. To stay healthy the application needs to set appropriate timeouts to limit the impact of flaky external services.

Why Bother With Timeouts?

Timeouts are an often overlooked setting when making network requests. We’ll be talking in terms of HTTP in this post, but timeouts apply to all network requests. But why should we care about setting timeout values?

Consider an application that makes an API call to an external service to translate some text. When a user clicks a translate button, the client does an AJAX request to our server, and our server calls the translation service. The API call takes an average of 200ms to return, so we’re able to return translated text to the user within around 400ms total.

Suddenly, the translation service performance degrades, and starts taking 5 seconds to return instead of 200ms. Requests to our AJAX endpoint start piling up, and now our web server starts to run out of available connections. Now our users start to see errors that the site is down! We can quickly try to restart servers, but the requests keep piling up because the translation service keeps taking our available connections.

How do we avoid this nightmare scenario? The first step is to set timeouts on all your HTTP requests. Yes, in this scenario, that means that users would see an error when they click the translate button, but the whole site wouldn’t crash.

How do HTTP clients handle timeouts?

Here are the default timeouts for Net::HTTP, Ruby’s standard library HTTP client.

@open_timeout = 60
@read_timeout = 60
@write_timeout = 60

(source)

The default for each of these is 60 seconds! Imagine if one of your third-party integrations started taking 60 seconds for each call. How would that affect your application? Notice that there are three timeouts here.

Open timeout: The maximum time allowed to open a TCP connection to the host
Read timeout: The maximum time to wait for an HTTP response
Write timeout: The maximum time to send your HTTP request

At a low level, this is how timeouts are implemented for most HTTP client libraries. For example, here is OkHttp for Java/Kotlin, whose timeouts default to 10 seconds.

internal var connectTimeout = 10_000
internal var readTimeout = 10_000
internal var writeTimeout = 10_000

(source)

Regardless of the defaults for your HTTP client, they will likely not match the usage pattern for your request. You need to be aware of each of these and adjust them appropriately.

Be Explicit

Whether you expect an HTTP request to take a long or short time, you should be explicit about that expectation. Set your timeouts to match your usage.

res = Net::HTTP.start(
  host,
  port,
  open_timeout: 1.0,
  read_timeout: 2.5,
  write_timeout: 1.5
)

Some HTTP clients try to abstract away the complexity of the three different timeouts. For example, here is HTTParty for Ruby.

if add_timeout?(options[:timeout])
  http.open_timeout = options[:timeout]
  http.read_timeout = options[:timeout]

  from_ruby_version('2.6.0', option: :write_timeout, warn: false) do
    http.write_timeout = options[:timeout]
  end
end

(source)

It lets you write this

HTTParty.get(url, timeout: 3)

But this is a deceptive API and easily causes mistakes. When reading this, you’d expect that the whole request can take up to 3 seconds. However, the actual effect is that each individual timeout is set to 3 seconds, adding up to a worst-case request time of 9 seconds (there’s actually more that contributes to total request time).

Base Your Timeouts on Data

At ParentSquare, we use Datadog’s APM product to trace applications. Tools like this or NewRelic’s APM product provide samples of each outgoing HTTP request along with the total request time. We can use this data as a baseline for choosing timeout values.

For example, if we know that the average request time for a certain endpoint is 100ms, and the max known request time is 300ms, we might choose 600ms for each of the connect, read, and write timeout values to give us plenty of buffer. It’s a good idea to build in some buffer in case the service quality of the endpoint degrades. However, you don’t want too much buffer since that could start to cause cascading failures in your application.

As you might have noticed, this requires some amount of educated guessing, since so far, we have no data that tells us how much of the request is taken by connect/reads/writes. Can we do a better job of choosing more accurate values?

While I’m certainly not an expert at this, I found this feature of cURL described by Joseph Scott in “Timing Details With cURL”.

First create a format file we’ll call curl-format.txt.

time_namelookup: %{time_namelookup}\n
time_connect: %{time_connect}\n
time_appconnect: %{time_appconnect}\n
time_pretransfer: %{time_pretransfer}\n
time_redirect: %{time_redirect}\n
time_starttransfer: %{time_starttransfer}\n
———\n
time_total: %{time_total}\n

Then run cURL with the -w or --write-out option, passing in our format file. Let’s just run it on the ParentSquare home page.

curl -L -w "@curl-format.txt" -o /dev/null -s  https://parentsquare.com

time_namelookup: 0.004379
time_connect: 0.072605
time_appconnect: 0.223555
time_pretransfer: 0.223701
time_redirect: 0.291480
time_starttransfer: 0.294933
———
time_total: 0.821796

This actually gives us a pretty good idea of the timings. The cURL manpage gives us a good description of each of these values:

time_namelookup The time, in seconds, it took from the start until the name resolving was completed.

Name resolution took 4ms, so we know that the DNS lookup portion of this request was negligible.

time_connect The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.

It took 72ms to complete the TCP connection. This would be included in open_timeout in our Ruby example above.

time_appconnect The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed. (Added in 7.19.0)

It took 223ms to open a TCP connection and complete the SSL/TLS handshake.

time_pretransfer The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.

In this test the request took 223ms before starting any data transfer. So far we’ve just opened a TCP connection and completed the TLS handshake. We haven’t sent our HTTP request yet.

time_redirect The time, in seconds, it took for all redirection steps including name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections. (Added in 7.12.3)

In this case, we did follow a redirect, and so far, after negotiating the redirect, 291ms have elapsed in total

time_starttransfer The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.

At the 294ms point, we’re starting to transfer our first byte. After that, the rest of the time is spend writing our request and reading the response. As far as I know, cURL has no way to differentiate between those times.

time_total The total time, in seconds, that the full operation lasted.

The total time for this request was 821ms after we received our response.

What does all this tell us? If we strip away the extra information, we see that we can get the information we need from cURL with two metrics, time_pretransfer and time_total, so let’s rerun the command. We don’t even need the format file now.

curl -L -w "Pre: %{time_pretransfer}\nTotal: %{time_total}\n" -o /dev/null -s  https://parentsquare.com

Pre: 0.223701
Total: 0.821796

Now we can make better educated guesses about our open_timeout, read_timeout, and write_timeout values.

Open Timeout

Our test data shows 223ms, so an open timeout of 1 second should give the endpoint more than enough room to occasionally respond slowly without timing out. Depending on your use-case, you may want to be more or less strict with this.

Read Timeout

If we subtract our pretransfer time from the total, we get 598ms. This includes both write and read time. In most real-world use-cases, your read time is going to take the most time, since it includes the entire time from when the server receives your request, to when you receive a response back. This will include time for the remote server to process your request and send the response. So we can assume that most of this 598ms was included in the read wait time. If you know of a better way of measuring this, let me know.

Write Timeout

In this case, we can set this to a fairly low value, since we’re assuming that most of the request time is taken by the read. However, in cases where you’re writing a lot of data (for example file uploads), you’ll need to allow time for that data to be transferred.

Experiment

Ultimately, the performance of your external dependencies may vary, and you might need to do what amounts to “guess and check” to tune in the right timeout values. Your application should be able to tolerate some timeouts without causing usability issues for your users. If it can’t, you will also need to add other forms of fault-tolerance (like retries) to make sure that your application can recover from a temporary service interruption in its dependencies.

Going Further

Even with our timeouts, we still might worry about cascading failures. Consider if we’re processing a job 10,000 times a minute, and each of those makes an HTTP request. That request normally takes 300ms, and we have our timeouts set around 1 second. If that request starts timing out, suddenly that job is taking 3 times longer than normal! This application likely won’t behave well if that happens.

To solve this problem, we need something like circuit breakers, as described by Michael T. Nygard in his book “Release It!: Design and Deploy Production-Ready Software”. I highly recommend reading it for great examples of catastrophic software failures and ways to avoid them.

I also wrote a library called Faulty that implements circuit breakers for Ruby.

Timeouts are an important first step to ensure your application is safe from failing HTTP endpoints. They are often overlooked by even seasoned developers. Make sure you think about them before it causes a problem.

A Portable Shell Script Template

Mon, 31 Jul 2017 00:00:00 +0000

Often write small utility scripts to perform repetitive tasks. More often than not, throw-away scripts become tools I come back to over and over. I’ve decided to spend a bit more time on polish so that when I use the script again, it is documented and easy to use.

Most of my scripts are written in portable shell code. I’ve found that shell scripts are best for writing small workflow tools. I always start with the same basic structure.

#!/usr/bin/env sh

usage() {
  cat >&2 <<EOF
Usage: example-script [OPTIONS] ARG1 ARG2...
Perform an example function

OPTIONS
-h       Help - Show this help message
-m VAL   My flag - Do something fun!
-n       Dry run - Print debug messaging instead of performing the function
-v       Verbose - Print verbose messaging
EOF
}

# msg(message...)
# Print a message to stderr
msg() {
  printf '%s\n' "$*" >&2
}

# verbose(message...)
# Print a message only in verbose mode
verbose() {
  [ -n "$verbose" ] && msg "$@"
}

# error_exit(message...)
# Print an error, then exit
error_exit() {
  msg "$@"
  exit 1
}

# usage_error(message...)
# Print an error and usage information, then exit
usage_error() {
  [ "$#" -gt 0 ] && printf '%s\n\n' "$*" >&2
  usage
  exit 1
}

while getopts :hm:nv opt; do
  case "$opt" in
    h) usage && exit ;;
    m) my_flag="$OPTARG" ;;
    n) dry_run=1 ;;
    v) verbose=1 ;;
    \?) usage_error "Invalid option -${OPTARG}" ;;
    :) usage_error "Missing value for option -${OPTARG}" ;;
  esac
done
shift $((OPTIND - 1))

[ "$#" -ge 2 ] || usage_error 'At least 2 arguments are required'

# Main script here

verbose 'Prints a message only in verbose mode'

if [ -z "$dry_run" ]; then
  echo 'This will not run if dry run is set'
fi

Step by Step

#!/usr/bin/env sh

This is the “shebang”. It tells the operating system to run the script with the sh command. I use POSIX compatible sh rather than bash for portability, although I sometimes prefer bash for some features (like arrays).

usage() {
  cat >&2 <<EOF
Usage: example-script [OPTIONS] ARG1 ARG2...
Perform an example function

OPTIONS
-h       Help - Show this help message
-n       Dry run - Print debug messaging instead of performing the function
-v       Verbose - Print verbose messaging
EOF
}

This function just prints a block of help text. I’ve learned not to skip this. Even for scripts I only use myself, I reference this help block constantly.

# msg(message...)
# Print a message to stderr
msg() {
  printf '%s\n' "$*" >&2
}

This function is basically just echo, but it prints to stderr instead of stdout. It’s always a good idea to print messaging to stderr. This allows the real output to be parsed by another script if necessary.

# verbose(message...)
# Print a message only in verbose mode
verbose() {
  [ -n "$verbose" ] && msg "$@"
}

This function is just like msg except it only prints if the -v verbose flag is set. I don’t always use this, but it’s good for situations where you sometimes want debug output.

# error_exit(message...)
# Print an error, then exit
error_exit() {
  msg "$@"
  exit 1
}

# usage_error(message...)
# Print an error and usage information, then exit
usage_error() {
  [ "$#" -gt 0 ] && printf '%s\n\n' "$*" >&2
  usage
  exit 1
}

These are just shortcuts for printing an error message and exiting. I use error_exit when encountering a runtime error, and usage_error when the user inputs invalid information.

while getopts :hm:nv opt; do
  case "$opt" in
    h) usage && exit ;;
    n) dry_run=1 ;;
    m) my_flag="$OPTARG" ;;
    v) verbose=1 ;;
    \?) usage_error "Invalid option -${OPTARG}" ;;
    :) usage_error "Missing value for option -${OPTARG}" ;;
  esac
done
shift $((OPTIND - 1))

This part collects option flags (like -n). For more information see man getopts. You can add or remove flags in this section.

The first line defines a loop over every flag. getopts is a utility that gets the next flag on each call. The first argument sets the expected flags, and the second is the variable name to store each flag in.

The string :hm:nv defines the flags. If the string begins with a :, getopts runs in quiet mode, meaning it won’t output any text. Each letter after that is an option. If the option is followed by a :, it accepts an argument. The -m option here accepts an argument.

The case statement processes each flag and sets variables if the flag is encountered. The $opt variable is the flag letter and the $OPTARG variable is the argument if one is set. A ? is given if an invalid option is encountered. A : is given if a flag expected an argument but it was missing.

The last line shift $((OPTIND - 1)) removes the option flags from the argument list so that $1 - $9, $@, and $* are set to the remaining arguments.

Now to write some scripts!

How to Fail Better

Sat, 08 Nov 2014 10:00:00 +0000

Software development is like an art. Every project starts with a concept, an idea that organically unfolds into something beautiful. That is the creative process that makes my work worthwhile. However, for every masterpiece there are a thousand failures. How should software developers deal with failure?

Fail Early

Failures are unavoidable. Rather than fighting against failure, software developers should embrace them as a useful part of the craft. In order to profit from failure, it needs to be expected.

This last week I was building a JavaScript library for managing data in a single page application. I began the project with the goal of keeping it simple. It turned out anything but simple. As the week went by I would come across a problem and solve it by adding a bit to the scope of my library. By Tuesday, the library had a dependency graph of all the tracked entities, and was managing object relationships. I was building an ORM. I recognized the warning signs, but I was committed. I had already spent two days working on it. By Thursday, it became obvious that my project had long ago missed my goal of simplicity and was headed nowhere. I had failed.

Why did I wait so long before admitting failure? I don’t think it was pride or fear. It just caught me off guard. I wasn’t expecting to fail. This blind spot is a dangerous one. This mistake cost my company four days of work. It could have been much more. Had I been watching for failure and heeded the warning signs, I could have caught my mistake much earlier.

In order to fail profitably, mistakes need to be caught early. The later they happen, the more they cost. There are two ways to do that. First, continually evaluate your work. Have benchmarks for success and failure. Are you meeting your original goals? Are you fulfilling user requirements with your work? Second, be accountable to someone. Often we cannot see our own blind spots. Be completely transparent about what you are working on with your co-workers. If you are self-employed, have a mentor or partner that you can share your work with.

Don’t Feed a Failure

In economics, sunk costs are costs that are already incurred. The time you have already spent on a project is a sunk cost. You cannot get that time back. The sunk cost fallacy is making a choice based on what you have already paid rather than objectively evaluating the current situation. When I saw the warning signs that my JavaScript project was not working, I rationalized it by saying that “I had already spent two days working on it”. That is a fallacy. Instead, I should have objectively evaluated my project and seen that it was not worth spending any more time on. Instead, I spent two more days on it before conceding. If a project or task has failed, kill it immediately and spend your time on a better solution.

Don’t Repeat Yourself

Failures are rich with useful information that often goes to waste. DRY (Don’t Repeat Yourself) is a principle used in software development to minimize duplicated code and redundant systems, but it can be extended even further to the development process. When I make a mistake, I evaluate it, find out why it happened, then take steps to prevent it.

After recovering from the failure of my project this week, I identified two reasons for the failure. I didn’t identify the problem soon enough, and my original design was too complex. To solve the early identification problem, I am using daily stand up meetings and more frequent code reviews to be as transparent as possible about my work. To improve my designs, I plan to collaborate more with co-workers during design stage.

No one enjoys their failures, but when we harness their power, failures can become a useful tool in the software developer’s toolbox. We use them to improve ourselves. If software development is an art, then our failures are just the road to creating a masterpiece.

How I Set Up GitHub Pages

Wed, 05 Nov 2014 21:00:00 +0000

I was suffering from analysis paralysis trying to choose a platform for this blog. Out of the hundreds of options, GitHub Pages and Jekyll provided the right balance power and simplicity. I’ll explain why, and how to get started.

What Is GitHub Pages?

GitHub Pages is a static web host integrated with GitHub so that you can deploy a website simply by pushing to your git repository. You can see the source code of this site at its GitHub repo. This post is written in markdown and its HTML was generated by Jekyll. GitHub has Jekyll built in, so as soon as I push a new post, the site is immediately regenerated.

Setting Up The Repository

First you need to create a new github repository with the name username.github.io. Replace username with your GitHub username. Don’t add a README, .gitignore, or license yet.

Now clone the repository on your machine with

git clone git@github.com:username/username.github.io.git

Now we’re going to set up Jekyll. You will need to have ruby installed.

gem install jekyll

cd username.github.io
jekyll new .
jekyll serve --watch

Jekyll is now running your site! Go to localhost:4000 in your browser to see the basic template that was created. If you make changes to any of the content files you will see the changes reflected in the generated site. Take a look at the Jekyll Documentation for more information on configuring and building your site.

After you have your site configured and you added your content, push it to GitHub.

git push

After your first push, it will take a while for your site to show up. For me it took about 15 minutes. Once it is created, you can see it at username.github.io.

Setting Up Your Domain Name

GitHub fully supports pointing your own domain name at a GitHub pages site. They go into detail in the help doc on the subject. I’ll explain how justinhoward.org is set up.

There are two components to setting up justinhoward.org to point to justinhoward.github.io. First, I need to set up my DNS records, then I need to set up a file called CNAME in the repository.

My DNS records look like this:

@    A        192.30.252.153
@    A        192.30.252.154
www  CNAME    justinhoward.github.io.

The @ records represent the apex domain justinhoward.org. Apex domains cannot be aliased with CNAME. We need to use A records for those. Make sure you are using the latest IPs from the GitHub documentation. If you are only using subdomains, you can ignore those records. The subdomain www is pointed directly at justinhoward.github.io with a CNAME (alias) record.

Now justinhoward.org and www.justinhoward.org will both go to the GitHub Pages site. However, GitHub doesn’t know which domain to prefer. Try typing justinhoward.org into your browser. You’ll see it gets redirected to www.justinhoward.org. To do that, you need to have a CNAME file in the root of your repository.

The file should contain the domain you want as your primary domain.

www.justinhoward.org

Next Steps

You can now set up a blog with GitHub Pages. What next? How about adding some comments? You’ll see that the comments on this site are powered by Disqus.

You’ll also want to customize your template. At the time of this writing, I’m still running the default Jekyll template. I’m planning to personalize it soon. Keep an eye out for that.