Sketch resources

It feels like I’m always tweeting about Sketch—mostly because I love it so much. I haven’t opened Illustrator or Photoshop in many months, a fact that makes me happy on a daily basis. I wanted to have a good running list of resources that I come across, so I’m hoping that this will grow into that. It will likely be small to start, but should grow over time. I’ll post updates as it fills out.

Surviving a HackerNews traffic spike on a $5/mo host

The two alternate titles of this post are Everything I know about running your own VPS or Host a pile of your high-traffic WordPress sites for just $5/mo, but let’s go with what we have.

A few months back, after scrolling over enough promoted tweets from Digital Ocean in my timeline, I got curious enough to spend the $5 for the basic plan and sign up. At that point I was simply interested in seeing what they had to offer — see what their interface was like, see what WordPress felt like on an SSD, etc. I certainly wasn’t intending on spending hours and hours tuning and tweaking and configuring, but yet that’s exactly what I did.

Surviving a Hacker News spike

When Throwing in the Towel on Programming hit Hacker News, I was at home cooking supper and playing with Jack. I happened to check in on the traffic about half-way through the spike and could see that things were humming along perfectly well. In the end, I had almost 10k visits in the span of less than 2 hours. Not huge traffic by any stretch, but more than most small blogs like this get in the run of a few hours. In the time since, I’ve done all kinds of additional tweaking and tuning, and I think at this point I could serve much much more traffic without the server breaking a sweat.

But that’s the end of the story. Let’s start at the beginning!

Setting up your new VPS

I chose Digital Ocean (DO) because of that enticing $5 gateway. You could absolutely go with Linode as well — I have no preference and have used and liked both. Here’s what I did at DO:

  1. Install Ubuntu (12.04 in my case)
  2. Install the lamp stack I need to run WordPress
  3. Setup my dotfiles
  4. Install wpcli
  5. Setup a new virtual host for a WordPress site
  6. Install WordPress! (though I did this mostly through wpcli
  7. Profit!

And basically that was it, all except for step #7. Turns out that even on a fresh server install on a blazing fast SSD, things were moving extremely slowly. I was trying to import some .xml export files from some of my sites and MySQL kept falling over and needing to be restarted, and things generally were not going well. It made no sense to me — I had a brand new setup, on a very fast hard drive but my one single site was just crawling along. That’s when I hit #digitalocean on Freenode to get some help.

The good folks in #digitalocean were extremely friendly and helpful. Not all IRC channels are friendly to beginners—that’s an understatement—but DO’s certainly was. They helped me with my initial setup and are still helping me today with the more complicated stuff I run into. Before hitting a wall and giving up, you should definitely head over there and start asking for help.

Tweak and Tune

After a considerable amount of time spent reading and asking questions on IRC, I had a good sense of what I needed to do to make my server perform. Here’s roughly what I did:

  1. Tune Apache
  2. No, I mean, really tune Apache
  3. Spend an entire day reading about KeepAlive
  4. Keep reading about KeepAlive
  5. Tune MySQL
  6. Loose sleep over setting the ideal key_buffer_size
  7. Install MySQLTuner (you should let MySQL run for 24hrs or more before acting on the recommendations this gives you

After I went through all of this, performance really picked up. MySQL wasn’t falling over while doing relatively simple tasks, and things were generally much much faster. But how fast? How could I tell that everything was performing better?

Tools of the Trade

The hardest part of the learning curve to running your own VPS is getting a handle on the tools you have available to you. Understanding how much memory you have and how much you’re using at any one time is the big one, and that took me a while to catch on to. Basically there’s two main commands/tools that you’ll want to keep an eye on. Every morning when I get into work, I’ll open up two terminal windows and run two commands to see how things are doing:

1. top 

Just log in to your VPS and run the command ‘top’. That should fill your screen with a whole pile of info. Everything from how long your server has been “up”, how much memory you’re using, what processes are running and what % of CPU power they’re using. I like to sort the processes by %CPU or %MEM — do that by typing Shift+O then either n or k.

Here’s what that looks like :

top

2. free -m

Top gives you a sense of how much memory you’re using at any one time, but keeping an eye on free is a better way. Just type that into your terminal and see the how much memory you’re using. The output looks like this:

free-m

The line you’ll want to look at here is the second one:
-/+ buffers/cache:        339        655

This is telling me that I’m currently using 339MB of memory and have 655MB free. I’ve recently upgraded to the $10 plan, so that’s why I have 1024MB (1GB) of memory and not 512MB. But that’s what you’ll want to look at. Of course you’re not going to want to have to keep typing that free -m command every time you want to see how much memory you’re using, so you should watch it instead:

watch -n 10 free -m
That will run this command for you every 10 seconds.

Ok, so let’s recap where we are. We’ve got a lamp stack installed on Ubuntu, created a virtual host, got WordPress installed, and we’ve got Apache and MySQL all tuned up and ready for another Hacker News spike. The only thing is, how do we know we’re ready? How do we test that everything we’ve done has helped? I’m still struggling with this part a little myself, but I’ve come up with a few strategies.

Load Testing

I use two tools for testing:

Apache Bench

This is a system tool that will let you send traffic from your desktop to your server. To use it, logout of your VPS and try sending some requests along. I use something like this:

ab -n 1000 -c 50 http://waterstreetgm.org/

This simulates requests (1000) and concurrent users (50) to your site. The output looks like this:

apache-bench

That’s a lot of information there. A few key lines to look at are your Request per second and the percentages at the bottom. The 50% 1410 is telling me that 50% of the 1000 requests were served to my visitors in less than 1.4s, which isn’t too bad. I’d like to get the mean time per request down a little lower (1874ms or 1.8s) but it’s still pretty good.

Blitz.io

Blitz is an extremely neat tool. This does a more real-world benchmark than Apache Bench does. It rushes your site with traffic and gives you a report of how you did. My report currently looks like this:

blitz

Blitz is suggesting that I survived 463 hits/s, and that my site could handle 40M hits/day (!), which seems a bit insane to me. That said, this one rush sent 27,804 hits to my site in 60s and I handled that no sweat, so maybe I could handle 40M visits per day.

How do I know it was no sweat? While running this Blitz, I have two terminal windows open running free -m and top so I can see what’s happening. Here’s what that looked like during this rush:

blitz-top-free

It’s a little hard to see the output in the terminal windows there, but the top CPU-consuming application during this blitz was Varnish, and on the bottom you can see that during all this I was still only using around 364MB of memory—I noticed virtually no spike in memory use during this test. That’s pretty neat. I’m also this close to scrapping the Apache setup altogether and taking Nginx for a spin—but that’s a conversation for another day.

Oh yeah, I installed Varnish here too, forgot to mention that! Honestly, at this stage, I still don’t know enough about Varnish to comment on the performance gain. I know it’s supposed to be a magic unicorn for managing traffic, but I don’t know too much about it yet. What I do know is that when I finally got Varnish setup and running properly, I tailed my mysql log and did the Apache Bench and Blitz testing and found that there were no (zero!) requests going to MySQL. Whether that has to do with WP Super Cache or Varnish, I’m honestly not sure. I hope to read much more about this in the coming weeks.

And that’s about it! With a finely tuned lamp stack, you really can serve a huge pile of traffic from just a 512MB, $5 VPS. It will take some time and effort, but you’ll learn a ton along the way. I will point out one caveat with this though: the need for speed is addictive. I started out just wanting to see what Digital Ocean was like a month later I’m editing a Varnish vcl file and sweating page load times over 500ms. But the good news is that it’s actually a lot of fun along the way. Enjoy!

The messaging app of tomorrow

A few weeks back, Facebook bought the (poorly named) messaging app, WhatsApp, for a metric tonne of gold bars. I’d never used WhatsApp before the acquisition (and now that Facebook owns it, I’m extremely unlikely to), but my curiosity was piqued. I downloaded and installed it, opened it up and expected something different than what I found.

whatsapp

This bajillion dollar app really doesn’t do anything until I connect with all my friends and convince them to go download it too. I can’t even send a message (the only thing the app does) to anyone unless they also have the app installed. Talk about a bummer.

So I went about trying to convince some of my contacts to give it a try. I went to three key people in my rolodex:

  • My brother:  “Can’t install any new apps. I don’t know what my App Store password is. Forgot it some time last year.”  aaah, ok.
  • My friend Matthew: “Nope” oh well then 
  • My wife: “I’m breastfeeding now. Maybe later, but unlikely.”  hard to argue with that then

I get the problem that Whatsapp solves, and I’ve seen the absolutely insane numbers they’ve posted since starting just 5 years ago. They’re enormous in Europe, South America and in lots of places in the developing world, and basically, everywhere except North America. I get why it’s a huge thing there and not such a huge thing here in Canada or in the US. I get it—I really do.

So when I describe the perfect messaging system in a few minutes, I don’t want you to say, “Aaah….he’s from North America. He doesn’t have the messaging problems that other places have. He doesn’t get it.” I’ve thought a lot about this and while I think Whatsapp is a neat app, I really don’t think it’s the messaging app of the future.

This isn’t your Dad’s messaging app

The messaging app I want doesn’t exist yet. For all I know, it’s technically impossible to make. The messaging app I want is extremely simple in theory: an always sync’d, any-device messaging service. Think iMessage for any phone, anywhere.

The messaging app I want will let me receive a message from a Samsung or Blackberry or flip phone on my desktop, let me reply to it right from there, and be instantly sync’d to my iPhone, my Nexus7, my home laptop, and — this is the big one — anywhere via the cloud from a web browser.

The messaging system of the future doesn’t care about contacts or devices or phone carriers. You send or receive a message from any device, on any device. Having messages tied to your phone is silly when you think about it—you don’t just send messages from your phone, do you? No. You send messages all day long, from wherever you are. Your desktop, your laptop, your phone, your tablet, your in-laws computer, etc. Messages should be tied to you, not to your phone.

Whatsapp is pretty neat for what it does, but it isn’t the messaging app of the future. This always ready, always sync’d, any-device system is. Again, maybe it’s technically (or politically) impossible — it’s hard to imagine Apple, for example, opening up its iMessages to another vendor, but we can dream.

This is the system I want, the system of the future — all your messages, available all the time, on any device. The company that can make this happen will be worth far more than $19 billion. Now we can only hope that Facebook doesn’t end up buying it.

How to Quickly clean up your iPhone photo stream

Two weeks ago my iPhone had 2,817 photos in it. It now has just 1,918.

How did I delete almost a thousand blurry, out of focus photos? I certainly didn’t click the little trash can at the bottom of each one. Nor did I have to go through and select all the rejects from my huge stream. I used Image Capture, a simple import application that ships with every Mac.

Image Capture

Screen Shot 2014-03-04 at 11.14.58 AM

Just connect your phone to the computer, start Image Capture and you can see all the photos you’ve ever taken. To delete the ones you don’t want, just scroll through and command-click to select the rejects. When you’re done, just select the red circle delete button at the bottom and the photos will be removed from your phone. The beauty here is that you can sort by size too, so you can see all those huge videos you’ve taken.

But now my backup folder is out of sync

Right, so now you’ve tidied your iPhone library, but the backup folder on your computer still has all the photos you’ve ever taken. Of course that’s a simple fix now too: just delete all these photos and re-import the culled photos from your phone.

Throwing in the towel on becoming a programmer

Man learn Javascript, c. 1900.

Man learn Javascript, c. 1900.

I think ready to hang up my programmer skates. In fact, it seems more likely that I never had skates to begin with. In the past 5-10 years, I’ve attempted to learn to code in virtually all of the major web languages and environments, using all of the latest tools and and classes and tutorials—and I’ve failed miserably every single time.

I just stumbled across Dawn Casey’s omg! I’m a n00b and too afraid to start. It is unbelievably good. Go read it, I’ll wait. Some parts are sad, some parts are laugh-out-loud funny, and I’m sure there are some parts in the middle there that are encouraging to beginners, but I can see right through the whole thing.  The whole ebb and flow from “holy man, I’m completely lost” to “humm…I think I’m starting to finally get this!” is something I know very, very well. I’ve felt this addictive, though ultimately disappointing feeling many times.

We’ll have some fun reliving the agony in a moment, but first a word of background for the folks at home: although I can’t program I can definitely code. The distinction needs to be made there. I’ve been coding for years and have actually gotten pretty good at it. WordPress is my main tool of choice and I’ve gotten quite handy with it — I maintain my own starter theme (a fork / amalgamation of several projects),  I write all of my projects from scratch, I write (modify) all kinds of custom functions to make different parts work, and I generally know the ins and outs of building reasonably complex sites with WordPress. On top of that, I’m extremely comfortable in the command line and I use Git for almost everything I work on. All that is to say that  A) I’m not a beginner, and B) I’m not non-technical. But, as we’ll now learn, I’m absolutely not a programmer.

A Sense of an Endpoint or, The Trouble with Programming

Over the years I have tried to learn all the big players: PHP, Javascript, Ruby, Python, Perl, Java and Objective C. I have failed to learn all of these. It’s almost staggering to even write/realize this. Seven languages. Seven! And I completely and utterly failed at learning all of them. What’s the issue then? After all these years, I think I’ve finally come up with the answer: although the road to starting to learn all of these languages is manageable, they all have a brick wall at the end. Let’s look at the four I spent the most time with:

PHP

PHP is my “best” language, though that’s a dubious honour. It’s the one I’ve been playing with the longest, and the one I’m most comfortable in, given all my time spent with WordPress. PHP is (should be?) a great language to learn with because all of the environment stuff is taken care of for you—just download MAMP, stick some PHP tags into a document and off you go. This is the language I’ve definitely spent the most time with — I’ve done several courses, read several thick books, read literally zillions of tutorials. I’ve gone through lengthy tutorials where I create an object that has a PDO or something to access my fake eCommerce store in my fake database. Things have actually gone fairly well a few times with PHP, in that I’ve gotten fairly far along with the material, but it never lasts long. Pretty soon it’s midnight on a Tuesday and I’m trying to access a query string that was sent via $_POST and I get thinking, “You know? Life is waaaaaay too short for this”.

php-codes

I’m sorry…..what?

So, while starting with PHP is great, going beyond the basics has felt like solving a Rubik’s Cube with my toes. In the end, every single time, I’ve decided that there’s no way I’d ever want to build anything with PHP that I couldn’t already do much faster/easier/better with WordPress. And hence, I’ve given up.

Javascript

This is the real fun one! I’ve spent almost as much time with Javascript as I have with PHP. I started with jQuery (which I can use reasonably capably) and eventually worked backwards into plain Javascript. It’s actually a lot of fun, at first. The thing with Javascript is you have control over when things happen. This control is given to you by funny things called “callbacks”. Essentially, you use a function to call (or “callback”) another function. Let’s say for some reason you wanted to hide every image in on page for 10 seconds on page load. All you need to do is create a timer function that counts for 10 seconds and then call the image loading function as a callback to that. See? Fun!

The not fun part about Javascript is that brick wall I was talking about earlier. After spending a lot of time going through tutorials and courses and reading books and building little projects, I really felt like I was ready to take on the Javascript world. After you’ve got the basics down pat, the logical next step is browsing through the 500 TodoJS apps and spending a month trying to decide which MVC framework will suit you best. After you decide, it will take you another month to try and figure out what an MVC framework even does. I still don’t really know.

So, when people tell you that Javascript is the wave of the future, this is what they’re talking about. If you’re feeling pretty good about all the JS you know, just have a look at this:

js-codes

I’m sorry, ({ what })?

At first I thought, “Hummm, I’m really catching onto this Javascript stuff!” Everything’s an object, callbacks, hell, I even understood what the module pattern was and why it made sense to use it (to avoid this “spaghetti” business people talk about)! But turning all that knowledge into a working Backbone app? That felt like swimming in cement. The funny thing about Javascript is that I still can’t see a way for me bridge those two worlds. I simply can’t see how I can take all these fundamentals that I know about Javascript now, and scaffold it up enough so that something like Backbone even makes sense to me. Despite hours and hours and hours of work, getting to that “next level” with Javascript feels literally impossible.

Python

I don’t have a lot to say about Python, except that I know the promise from xkcd is an empty one. I tried it a few times, and worked through the course at Codecademy, but it never felt very natural. I didn’t spend too long with it, but it never really clicked. On top of that, there’s a constant din of “Python’s not for you, it’s for them ({scientists, academics, hackers, statisticians, someone else})” out there if you look up stuff about Python. The language has always been really appealing for me, but for better or worse it’s never felt like something I should invest my time in. That, combined with the fact that I discovered Ruby meant the end for Python.

Ruby

If you’ve never touched a single language, this is the one for you. It really is beautiful, like so many say. It’s short, concise, fun, productive, and a lot of it really, truly reads like English. Of all the languages, Ruby was by far the most natural and fun. In the months I spent learning Ruby the hard way, I’d run home from work to get back to it. I really loved working on new things in Ruby — they all just made sense so quickly. By the end, I was feeling so happy and comfortable with Ruby that I even tackled some problems in Project Euler with it.

But as I look back at it now, I could have easily used PHP or Javascript for those same problems. I still like the syntax of Ruby better than all the others, but I wasn’t doing anything with Ruby that I couldn’t do with PHP or JS. I’d write a clunky function that did something with x and y and returned some value at the end. Doing it with Ruby was fun, but I wasn’t doing anything beyond playing with the primitives. Well, what’s beyond the privatives you ask? In short: Rails. Just like all the others, Ruby has a brick wall as well. It’s called Rails. The only thing is, Rails isn’t just a brick wall, it’s a brick mountain.

A few years back, I spent about a month getting comfortable with Ruby. It was actually really nice, and as I’ve said, fun. Actual, legitimate fun. After I was happy with where I was with Ruby the language, I started in ernest with Michael Hartl’s famous Rails Tutorial. I probably lasted another 6 weeks after that, but I knew pretty early on that it really wasn’t going to happen for me. In the tutorial, Hartl introduces Ruby, Rails, Git, Heroku, Test Driven Development and just about everything else you can think of, right from the start. By the end of the first month, I had literally no idea what was going on. I’d rake something and then route something else, and then I’d try and migrate up to (or down from) somewhere and absolutely none of it made any sense. By quittin’ time (around week 6) I was 100% convinced that anything I’d end up building with Rails could be build in a fraction of the time in WordPress.

Sure, but what about Sinatra? I actually did a few projects with Sinatra as well, and really liked using it. It felt pretty fun too. Way less behind the scenes magic, and you could actually see what was going on. But what was the point? Rails is still the big endpoint, and playing around with Sinatra really doesn’t get you too far down the road to learning Rails.

A light at the beginning of the tunnel

The whole business of programming is extremely complex. There are so many moving parts to anything these days, and one could easily spend a year just learning the tools available for a given language/environment. Javascript is a perfect example of this—it’s completely and utterly obsessed with tools. Not that these tools aren’t useful, there’s just a million of them.

I’ll end with two excerpts. First, a quote from the omg I’m a noob piece that I mentioned at the top, that sums all of this up nicely for me:

incomprehensible

Second, probably the true impetus for writing this post, is a few lines from David DeSandro‘s ImagesLoaded Javascript plugin. While en route down a winding rabbit hole the other day, I stumbled across this plugin and took a second to look through the code to see if any of it made sense to me. By about line 20, I was actually laughing out loud. I have no clue whatsoever what the code is doing, despite all of my courses and books and hours spent at the screen. Here’s the selection that made me spit coffee onto my keyboard:

desandro-what

Really though, just what is happening here?

It’s not that I’m done trying to learn any of this, or certainly not that I don’t find any of it enjoyable. I suspect that in years to come, I’ll go pick up a fresh copy of Ruby again and spend a month or two using it to figure out some Project Euler problems, but I think that my ambitions to become a programmer have finally extinguished. And I think I’m actually relieved by this.

Here’s to spending more time outside!