Wednesday, 29 March 2017

Timing your background tasks in Ruby (or bash, or any other language...)

Is It Working runs a handful of critical tasks in the background.

This post will show you how I use the new timing feature in IsItWorking to make sure that they run in a reasonable time.

Background tasks do critical things like requesting checks for your ssl expiry date, or a new whois check on your domain.

They also do cleanup tasks like clearing out excessive records on your checkins.

I use Javan's whenever to manage the schedule. It generates a crontab whenever I push an update with Capistrano.

The schedule is really simple with a handful of entries like:

every 1.minute do
 runner "Switch.check_for_past_due";
end


This task looks for any checkins that are late. It marks them as late, and queues notifications to be sent as async tasks.

def self.check_for_past_due

  Switch.not_late.past_due.find_each do |switch|
     switch.late = true
     switch.save
     switch.user.notifications.each do |notif|
       notif.send_switch_is_late(switch)
     end
  end
 
end

At the moment, this task take a couple of tens of milliseconds, but as the number of users increases, it will slow down.

At some point, I'll need to do something new; Perhaps use a faster droplet, optimise the code, or figure out some other solution!

IsItWorking's new timing feature lets me easily monitor how long this takes, and let's me get a warning if it starts taking too long.

Step 1) Log in to IsItWorking and create a checkin.



I could use the 'no timeout' option - but as I know the script will be running every minute, I might as well get the alert if it fails to run after 15 mins.

I'm using the IsItWorkingInfo gem to keep the code simple (but if you want to skip the dependencies, it is super-easy to do manually)

I add the gem to my gemfile

gem 'is_it_working_info'

then install it

bundle install

click on the 'use' button to get my checkin id



I can ignore the url as I'm using the gem - but this is actually what the gem will be pinging.

Now I just wrap my critical code with the timing block

def self.check_for_past_due

 IsItWorkingInfo::Checkin.time(key:"MYUNIQUECODE",
    message:"Switch.check_for_past_due",
    boundary:1000) do

      Switch.not_late.past_due.find_each do |switch|
        switch.late = true
        switch.save
        switch.user.notifications.each do |notif|
          notif.send_switch_is_late(switch)
        end
      end
    end
 
end

I have added a message to make it easier to remember what I'm doing here.

The boundary time is set in my code, so if I ever need to change it, it is part of the checked in code in my project.

If the task ever takes more than 1000 milliseconds, then IsItWorking will alert me and I can investigate.

I can easily review the current timing at https://IsItWorking.info, and I can also download timing data as a csv for investigation.


Introducing Time alerts for your tasks

I'm pleased to announce the addition of a timing function to IsItWorking.info checkins.

These allow you to send the time your task took, along with the boundary time that you consider acceptable.

If the task took too long, then IsItWorking will alert you.


     
#!/bin/sh
start=`date +%s`
/Users/rob/Documents/Development/Rails/IsItWorking/test/scripts/slow_script.sh
end=`date +%s`
runtime=$((end-start))

curl -d "t=$runtime&b=5" https://api.IsItWorking.info/c/CHECKIN_IDENTIFIER


In this example, if slow_script.sh takes more than 5 seconds, you'll get an alert.

You can read more in the api

Or see other examples

Monday, 27 March 2017

After 10 years, Active-Domain messed up my domains

I thought I could trust my domain provider to give me reliable notification before my domains expired.

I was wrong.


While I was building Is It Working, I got an email from Active Domain reminding me to renew a handful of my domains

Nothing new here - Active Domain have been providing me with a splendidly boring service for about a decade.

I have always had multiple reminders before a domain expires

The difference this time was that one of my domains had already expired
Logging in to Active Domain, showed me that they had 'updated' their website to a fancy new system

Presumably, somewhere in the upgrade, they stopped sending out renewals, and my domain expired without notice

Fortunately it wasn't a critical domain, and it was only a few days past the expiry date, so I was able to recover it

Over the next few weeks, Active Domain's 'shiny new' system proceeded to apply the wrong contact details to several of my domains. (resulting in incorrect whois details, and difficulty transferring domains to a new registrar)

Their login stopped working for several days

They 'lost' a couple of my domains

I decided that I should add some simple domain monitoring to Is It Working!


Is It Working will now send you a notification if your domain is getting close to expiry:


It will also alert you if there is a change to your WHOIS record, and let you examine the changes as a diff

(extract below shows the whois change after ActiveDomain changed my contact details back to the right person!)


I have named Active-Domain as they had a whole list of mistakes - but the point is that this could happen anywhere. They were a company that I had used for years, and who I recommended to my friends. They were reliable, simple and boring.

I don't know why they decided to update their systems - but this kind of problem can happen anywhere an upgrade is botched.

IsItWorking.info is designed to provide early warning, and peace of mind in the future...

Welcome - Please let me know what you think.

Is It Working aims to provide simple cost effective monitoring for some aspects of your websites and servers.

I got fed up of yet another site asking for $9.99/month to provide straightforward monitoring. I figured that I could do a similar job for a tenth of the price!

Is It Working is the result.

At launch, Is It Working helps you to answer three questions.

1) Are my cron scripts (or other regular tasks) running correctly
2) Are my SSL Certificates up to date
3) Are my domains up to date (We'll also tell you if the WHOIS record changes)

Please try it out, and let me know what you think.