tail -n 50 articles.txt

I'm listed on quite a few startup websites as a "Technical Advisor" and when I attend company events there's often a bit of confusion around what I actually do.

Let's not focus on me though - I'm here to dispel some of the myths around a Technical advisor. 

Number one: They don't write code, or at least not a lot of it. People that write code are developers, engineers or founders.

Number two: They didn't (usually) put their money on the line. If they did they'd be an investor.

Number three: They don't replace a property CTO or technical founder as much as the founders wish they did.

So what does a Technical advisor actually do? Not to be drill but they literally do as the name implies, they advise the fingers on technical matters.

Like what you ask? Everything technical. Language questions, rails vs django, scaling and development best practices, hiring advice, code reviews, salaries, the right tech intros. That's where a tech advisor who knows his shit is worth their weight in options, all 0.x percent of them.

Now you might be thinking, isn't thar the role odd the CHI or tech founder. Sure, if you were lucky enough to hire or find a tech superstar who isn't so busy writing code to take care of all this stuff then congrats, you might very by without a tech advisor. But if you ever want a second opinion, some extra insight from someone who has done it all before then maybe those couple points of options don't sounds like such a high price to pay.



Weigh up the wait

Posted by Timothy Marks | 2015-01-16 | 0 Comments

The process of making a request to a http server is fairly well understood - the client or browser makes a request and waits, the server accepts the request, does some processing (file reads, database lookup, compute the answer to life the universe and everything) and returns a response to the waiting client.

In this view of the world let's make some assumptions to make things simpler.
  • The client is waiting for a successful response to allow the user to move onto some task
  • The response is indepedent of some of the work required by the request
  • Client timeouts and page loads are important, 1s makes all the difference

  • For our example let's look at a fairly simple and insecure example, a user is uploading a list of email addresses to send some pre-canned email to. The user wants to confirm that the server received the list of addresses but doesn't actually care if the email goes out now or in 1 minute.

    Let's look at a very generic example in rubocode (ruby and pseudocode)
    
    
    def send_emails()
    emails = request.emails
    emails.forEach do |email|
    # Wow sending an email is slow, it can take almost a second to send one
    mail.deliver(email, "Hi, you have a mail", "Big text email")
    end
    return "Sent all the emails, awesome work"
    end

    This example might work fine with 1, 5 or maybe even 10 emails. That comment looks scary though - does sending an email really take up to a second? That means that the simple task of sending 10 emails could delay a response to the client for more than 10 seconds.

    This sort of example would be perfect for performing in a background 'thread' (I use the term broadly). This pattern goes by a number of names - background tasks, job queues, async tasks, but fundamentally the idea is to remove the process of the email sends from the dependency of the response.

    Let's look at a simple way to do this (don't do this in production either)
    
    
    def send_emails()
    emails = request.emails
    emails.forEach do |email|
    # Wow sending an email is slow, it can take almost a second to send one
    Thread.new{mail.deliver(email, "Hi, you have a mail", "Big text email")}
    end
    return "Sent all the emails, awesome work"
    end

    Now since each mail.deliver is being performed in a seperate thread, our main thread can immediately move onto returning a response. This is bad for a multitude of reasons (threads can crash, the main program can crash, no visibility that a send succeeds) but it works, the endpoint is now really quick! Even sending 200 emails takes less than a second to return a response, even if sending the mails takes a lot longer.

    Async tasks use queues as a middleman and the worker pattern - code that receives a message and performs an action on the message content instead of threads. This allows you to build more infrastructure, monitoring, concurrency and retrying into your system as opposed to less transparent threads.

    We'll go further into async tasks in another post or in the meantime checkout sidekiq, celery project or sneakers!

    The art of Queueing

    Posted by Timothy Marks | 2015-01-14 | 0 Comments

    The art of queueing, it's not just the glue that keeps humanity from splitting apart at the seams in a brutal free for all for consumption (assuming you haven't watched a video of queueing in some countries maybe it's best not to at this point).

    Queues in tech provide an important glue between systems that alleviates the pain and management of mainly not having a queue there. Without a queue everything becomes a real time game of ask and wait and if there's too many asks then all you end up doing is waiting, a lot.

    Queues software or services can provide varying numbers of key tasks in a distributed system. Some common examples include
    • Delivery receipt: put a raw http request on a queue and you can confirm receipt to the client without having to perform any processing. Make retry and HA your backends problem, not the clients
    • Fanout: want one message to be at two places at once? Http works, but what happens if one succeeds and the other fails. Fanout queues ensure that receipt is ransactional and a message will be in two or no queues, never just one. It also scales to 50 queues or 100 queues with smaller overhead than most other methods.
    • Routing: well, some queues can do this but make consumption someone elses problem, take delivery logic out of the API.
    • Buffering: Get a sudden spike of API calls or some servers go down and capacity gets impacted, within reason a message sitting in a queue is safe and timeless. Raise or replace capacity and process everything after the fact, uptime dependencies jsut got smaller (hint it's now your queue but that's better than 50 services).

    If you think you've never used a queue before you may be surprised, an array with push and pop is as simplistic a message queue as you can get. Most message queues implement FIFO or first in, first out, where the messages are returned in order of insertion. If you're interested however in checking out some more production ready queues though, RabbitMQ is pretty much the industry standard for high volume message queues using a standard called AMQP. If you're cloudlike there's SNS and iron.io and Redis can be (ab)used as a queue if you're really keen.

    Fine wine and Programming Languages

    Posted by Timothy Marks | 2015-01-06 | 0 Comments

    Choosing a programming language for a project is like choosing the right wine to go wth dinner (or scotch, or soft drink or ice cream for that matter, pick your poison).

    Every language, yes even PHP, has the right set of features to excel under some cicrumstances and make a complete mess under others. And it's not even a question of choosing the right tool for the job because the right tool sounds so conclusive and in reality it's all a matter of subjectivity, taste and experience.

    Some of the 'slowest' or uncool languages have amazing community and frameworks that can easily make up for the language deficiencies. Rails isn't the fastest, most efficient or most attractive framework but being able to go from nothing to webapp in under and hour is just incredible. Python has it's proponents and detractors but the simplicity and expressiveness of Django and the power of the scientific modules is unsurpassed. Go has a struggling community and lack of package management tools but the stdlib and core focus enable incredibly fast, simple and stable system programs to flourish.

    So ignore the hype or hurt and test and try things continually. Try to push your boundaries next time and instead of playing it safe on a side project invent and innovate, even if that means trying Sinatra, or Flask for some pythonic movement, instead of Rails. What's the worst that can happen?

    Coming to you live from the past

    Posted by Timothy Marks | 2014-12-27 | 0 Comments

    Social media presence is difficult, in an always on, always connected world those that don’t automate starve their ever hungry audience of content and mindshare.

    We’re seeing more and more social automation these days, from simple customer retention messaging that sounds like it comes from a real person with a slightly interesting reply-to address using intercom to a continual drip of tweet after tweet from tools like Buffer.

    Well, we’re helping to bring blogging into the next stage of automation and step one of our long tail approach to having the best content and automation tools for blogging starts with the ability to queue and time post releases.

    So this post here is coming to you from an hour ago through the magic of technology.

    Enjoy and be sure to check out https://letterfully.com for more features and the best blogging experience through Evernote.

    The new TLD land grab

    Posted 2014-09-05 | 0 Comments

    I'd always had reservations about the current insanity over the new custom top level domains. Having a vanity .ninja domain seems harmless but the ever growing TLD space is putting added pressure on startups and trademarks. 

    Now in addition to buying out the .com .net and whatever else is the flavour of the month (look at poor old .ly now), look for squatters and opportunists registering yournewstartup.pizza just to be painful.

    There's an even more painful side that people are going to start running into. The old practice of abusing imaginary TLDs for internal URLs is now running into serious namespace collisions.

    Xfinity defaults the DNS domain to home.network a domain that until 4 days ago didn't resolve. Thanks to the launch of the .network TLD however, it now does. DNS now struggles to coalesce the real and virtual world and my MacBook suddenly starts failing DNS lookups for rubygems. Thanks Comcast!

    I'm sure this is just the beginning, it's no Y2K but as the land grab spills over into the previously abused fake domains no doubt we will see more and more wierd collisions and unforeseen behaviour.

    For now all you ruby Mac users can comment out the domain from /etc/resolv.conf but it's not exactly a fix to the underlying issue.

    Should we be mad at Xfinity for abusing the domain registry and taking something that they never owned under the assumption nobody else ever would? Sure, it's dumb and stupid but so is most technology and this doesn't feel like progress!

    Trying to Go

    Posted by Timothy Marks | 2014-07-23 | 0 Comments

    I’ve been playing around with a number of new languages to bootstrap my side projects. The latest was a serious attempt to bootstrap a social image site (with a twist) on Go. Here’s my take on the good, the bad and the ugly.

    The Good

    Go is fast, I don’t mean kind of fast or fast compared to everything else, I’m talking the kind of fast when the entire app compiles before I can type even a sentence of this blog post. Coming from some serious Scala work for the last Stripe CTF and waiting literally minutes between changes being able to build, test and run on every save is one of the biggest boons to my development flow since Atom.io

    I also love the raw simplicity of the Http handler, especially with simple Middleware libraries like gocraft/web. Injecting smart contexts into each route handler offers a simple way to manage dependencies, database connections, config, content-type handling without the bloat of most full-stack web frameworks.

    I also don’t mind the strict adherence to standards with gofmt and the heavily criticized var, err := func(). Enforcing error handling at the point of every call forms good habits instead of blowing exceptions all the way up the stack.

    The Bad

    Go, the community, is very immature. There is a significant lack of solid libraries for OAuth, Image processing and some other important things. This isn’t a complaint of the language and I’m doing my part to cleanup some of the OAuth libraries but it seriously impacts the ability to throw up a demo for an investor or partner.

    The Ugly

    Rendering JSON or just dealing with JSON feels painful when coming from more dynamic script based languages. Sometimes the Rails magic just works and makes handling content-types and serialization just work. Templating for anything that’s not a simple JSON/XML API is also a nightmare, would love to see consensus on handling haml/jade or any number of not having to write raw html templates.

    Go dependency management also seems hacked on, I am completely against the idea of bundling all my dependencies into my source control, bleeugh, please. Goop comes close but isn’t widely supported or particularly well integrated.

    Overall

    Go is an incredible language and can be used to bootstrap some insanely clean, fast API’s for webs services. Given some time (I’m putting some cash on the next year) I expect to see a set of tools emerge to manage the current issues I’m running into and make Go a first class web citizen. For now there’s hundreds of other languages and frameworks vying for developer mindshare and attention. For now though, Rails seems to be my go-to to bootstrap site in 3 days and scale to the point of a future rewrite under the glorious pain of success.

    Great things come in beautiful packages

    Posted by Timothy Marks | 2014-05-22 | 0 Comments

    It’s no secret that product design is hot right now. Look no further than the immense success of Apple and the emergence of highly backed product projects on Kickstarter. Having a clever or quirkily designed product captures the consumers imagination and sets the immediate desire for a product.

    There’s another saying, you only get one chance to make a first impression. With online fulfilment and ‘pre-release’ backing, the first experience most customers have with a product is the second they tear open the postal packaging. You only get one first impression, and these days it’s a box.

    It’s no secret that Apple was a trailblazer in sexy packaging. Their white, minimal boxes promised the utmost pleasure as the top slides off and reveals the perfectly packed, not a single cm of space wasted caress of your new product. The unboxing experience became almost as rewarding and lusted for as the product itself.

    That’s why I’m becoming more amazed at the detail, time and thought to turn what used to be waste and packaging into an unboxing experience.

    Here are some great examples of a brilliant experience you can order at home. I won’t add the iPhone and I don’t have any pictures but the Leap Motion unboxing was great as well.

    A very different shape and size, cleverly branded without being too blatant.




    Clever, unobtrusive, classy. Love the raw look and feel. 


    I’ve recently been playing around with Geolocation on Mobile again. I’m picking a trend now, geo will be one of the next big thing in we’ll see pushed as the new cool in iOS and Android apps and games.

    I thought that while I wait for the trend to catch on I should bring myself up to speed with the new (well, as in 2.4) Geo indexes in MongoDB. My last foray into location based databases was Elasticsearch and before that Postgis so I was interested to see how Mongo stacks up.

    This isn’t intended to be a benchmark of any of the systems, or a detailed comparison (on why I’m beginning to detest the instability of Elasticsearch) but to document the steps to go from no to geo.

    First we need to create a collection and setup the GeoIndex.

    
    
    db.geoexample.ensureIndex( {"location": "2dsphere"} )
         
    We could use just a 2d index but I prefer to be more ‘correct’ and use proper spherical distance between GPS points.

    Then we need to add some example data to the collection. Mongo uses standard geoJSON so lets add somewhere interesting, how about the Golden Gate Bridge.

    
    
    db.geoexample.insert( { 
      "place": "Golden Gate Bridge”, 
      "location": { "type": "Point", "coordinates": [-122.477502,  37.810996] }
    } )

    Something to note is that Mongo and geoJSON use the format long, lat instead of the more common (and Google Maps) lat, long

    Now that we have a location, lets pass through some here I am data to see what’s around me, first lets try searching in SOMA for places within 2km (2000m).

    
    
    db.geoexample.find( { "location": { 
      "$nearSphere": { "$geometry": { 
        "type": "Point", "coordinates": [-122.408909, 37.779059]}, "$maxDistance": 2000 }
       }
    } )

    We get nothing back unsurprisingly, SOMA is at least 5km away from the Golden Gate Bridge. This time, lets try with a max distance of 10km.

    
    
    db.geoexample.find( { "location": { "$nearSphere": { "$geometry": { 
      "type": "Point", "coordinates": [-122.408909, 37.779059]}, "$maxDistance": 10000 }}
    } )

    This time we get the Golden Gate Bridge back incredibly quickly.

    
    
    { "_id" : ObjectId("528772d7218c9a746c3b57a6"), 
      "place" : "Golden Gate Bridge", "location" : { 
      "type" : "Point", "coordinates" : [-122.477502,  37.810996 ]
    } }

    So far, Mongo looks like a very simple way to deal with Geolocation data. Turns out you can also run "$near” queries on a 2dsphere index as well so you can chop and choose regardless of the index! However, so far we’ve only been getting a list of what’s nearby, how about how far away it is? Is that possible, the object it returned is exactly what we saved in the database with no ‘magic’ attributes.

    Turns out there is a way, my favourite Aggregation frameworks to the rescue again.

    
    
    db.geoexample.aggregate( [{ "$geoNear": { 
      "near": [-122.408909, 37.779059], 
      "distanceField": "howFarAwayIsIt”, 
      "maxDistance": 10000, 
      "spherical": true }
    } ] )

    Returning the magical value

    
    
    {
      "result" : [
         {
           "_id" : ObjectId("528772d7218c9a746c3b57a6"),
           "place" : "Golden Gate Bridge",
           "location" : {
             "type" : "Point",
             "coordinates" : [
               -122.477502,
               37.810996
             ]
           },
           "howFarAwayIsIt" : 0.0010980200058842591
         }
       ],
      "ok" : 1
    }

    There are a couple of points of interest here! 

    Firstly, for aggregation queries you need to specify a spherical variable, based on whether your index was set as ‘2d’ or ‘2dsphere’. Unlike normal queries, the aggregation query will throw an exception if you specify spherical false (default) on a 2dsphere index or vice versa.

    Secondly, the distance returned looks all sorts of weird! 1 mm away? No way, turns out the aggregation framework defaults distance to radians, even if it uses metres for all the other distances (such as maxDistance). Don’t worry, there’s an easy fix. We can provide a distance multiplier. Doing some research we find that The radius of the Earth is approximately 3,959 miles or 6,371 kilometres.

    So lets run the query again but this time to get a result with a distance in km.

    
    
    db.geoexample.aggregate( [{ "$geoNear": { 
      "near": [-122.408909, 37.779059], 
      "distanceField": "howFarAwayIsIt”, 
      "maxDistance": 10000, 
      "spherical": true, 
      "distanceMultiplier": 6371 }
    } ] )

    And we get

    
    
    {
      "result" : [
        {
          "_id" : ObjectId("528772d7218c9a746c3b57a6"),
          "place" : "Golden Gate Bridge",
          "location" : {
            "type" : "Point",
            "coordinates" : [
              -122.477502,
              37.810996
            ]
          },
          "howFarAwayIsIt" : 6.9954854574886145
        }
      ],
      "ok" : 1
    }

    Which gives us a fairly cool distance of 6.99km. Google maps confirms this is pretty accurate so there we go! All in all, it’s pretty damn simple to run your own Four Square clone (minus all the locations) on Mongo in minutes!

    I’ve been using Rails for a few years and had heard numerous ‘best practices’ for arranging page specific javascript bindings. I’d tried Paloma but found too much overhead and code generation.

    Then recently I stumbled upon This rails article. I highly recommend a read all the way through before coming to a conclusion. However, nested in the middle I found a link to the following, Simple JS Rails Controller Pattern. I love the simplicity and am happy to wear the cost of body data attributes.

    This got me thinking, I’ve always disliked seeing Rails Coffeescript files, each containing their own Jquery.ready listeners and thought there must be a simpler way. Extending the above pattern, I added a shared array to the global object in Appname.js

    this.Appname ?= {}
    this.Appname.shared = []
    

    Then we create the shared classes that should run on every controller and action.

    class SomeSharedClass
    
      init: () ->
        console.log('Called shared function')
    
    Appname.shared.push(new SomeSharedClass())
    

    Then we add a function to iterate each shared from the global and run the init() function similar to the controller and action pattern within our app.js

    (function($, undefined) {
      $(function() {
        var $body = $("body")
        var controller = $body.data("controller").replace(/\//g, "_");
        var action = $body.data("action");
    
        var activeController = Appname[controller];
    
        if (activeController !== undefined) {
          if ($.isFunction(activeController.init)) {
            activeController.init();
          }
    
          if ($.isFunction(activeController[action])) {
            activeController[action]();
          }
        }
    
        var shared = AppName.shared;
        $.each(shared, function(i, v) {
          v.init();
        });
    
      });
    })(jQuery); 
    
    This ensures that we have only one Jquery on ready listener and execute both the controller and action specific functions but also any global shared functions. It also maintains load order and dependency management ensuring all