Don Don's Garage

Adding to the slowness of the internet a little bit each day.

  • Cars
  • 57 Wrecker
  • Bluebird
  • Programming
  • WebROaR isn't ready for production, yet.

    • 6 Dec 2009
    • 1 Response
    •  views
    • programming
    • Edit
    • Delete
    • Tags
    • Autopost

    WebROaR http://webroar.in/, a Ruby web application server designed to support Rails and Rack based applications recently came to my attention. My currently preferred Rails stack is Apache2 http://httpd.apache.org/, Ruby Enterprise 1.87 http://www.rubyenterpriseedition.com/, and Passenger http://www.modrails.com/, using NewRelic RPM http://www.newrelic.com/features.html for notifications and metrics.  I wanted to see how WebROaR stacked up as a replacement.

    I began my investigation the way I do any application. I start by writing an installation script.  My tool of choice for deployments right now is Capistrano http://www.capify.org, and I've recently been interested in Deprec http://deprec.failmode.com/, a framework of recipes built on top of Capistrano specifically designed to install on ubuntu, which by chance, is also my preferred linux distribution. So I decided to extend Deprec with a WebROaR installation recipe.

    WebROaR began showing it's relative youth immediately.  I expect to be able to install a production application non-interactively. That is to say, I should be able to script the entire affair by passing values to the installer, configure, and make scripts via the command line.  The current version of WebROaR's install will prompt you for up to four different values and provides no mechanism to create a non-interactive install.

    With Capistrano a non-interactive install can be worked around by examining output and responding as a user would with stored answers. Other deployment systems based on shell scripts will find installing WebROaR challenging.

    The next issue to deal with is provisioning the application.  There was no mechanism, save rewriting the entire configuration file, to create a new application from Capistrano.  For this reason I left any configuration of the server or applications out of the deprec recipe.

    The problem with re-creating the entire configuration via Capistrano is a simple one, when you install an application with Capistrano it doesn't normally have any concept of any other applications that may be hosted by your server.  So a rewrite of the entire config would end up wiping out all other applications unless pains were taken to make it tolerant.

    Another sign that WebROaR is fairly young is that when I deployed a simple test application, the deploy scripts hadn't properly used the bundle command to unpack all of the application's dependent gems. It left the application in a really sorry state.  WebROaR's user interface didn't expose the error condition, nor was it able to serve the configured application.  It wasn't until I connected to the box via ssh and tried to run script/server in the applications deploy folder before I got any clue as to what the problem might be.  WebROaR should have done what it could to raise the problem to my attention within its exceptions section. I dealt with several errors where the WebROaR UI wasn't any help which gives me the impression there are many areas where WebROaR is rough around the reliability edges.

    It's great that WebROaR's concept is built around interactive management, Apache, Passenger, Mongrel, and Thin don't really provide any instrumentation without something like NewRelic's RPM.

    The exception reporting was definately useful, and presented nicely, if not exactly timely.  After triggering a few errors it still took several refreshes of WebROaR's UI before the information was ready to be examined.  Similarly there was a delay from when the first requests were generated before the graphs in the analytics could be reviewed.

    Things like email and sms notifications are not in the product yet. I would have expected the ability to automatically send notifications and have some control over the process when metrics go above or beyond certain thresholds, or when certain types of exceptions occur.

    The admin interface is protected by only a single username and password, multiple users aren't possible. Obviously role based access controls aren't in the product yet either.

    The documentation is rather good for what features there are. But there aren't that many features.

    I didn't do any kind of exhaustive performance benchmarks or stress tests because my curiosity had already been sated by getting my test application up. WebROaR doesn't compare to the configurability, stability, and breadth of options that I could even remotely consider it as a replacement for my current Rails Stack, even if all of it's claims about performance are true.

    If it's going to become a production ready application, it's going to have to make it effecient for sysadmins to install and manage it from the command line and report error conditions better.  I can absolutely see it's alure to hosting companies as a great way to host rails apps if they can delegate control to end users and use an API to control the configuration. The project is very young, announced November 25th, 2009 according to their Blog, and it's incredibly promising, I'll definately check it out again when they get near a 1.0 release.


    Deprec WebROaR recipe: http://github.com/donnoman/deprec/blob/2cdac7ed5c322d41512673a2e199707b3e47de...

    Related changes to deprec: http://github.com/donnoman/deprec/commit/2cdac7ed5c322d41512673a2e199707b3e47...

    Any fixes will be comitted here: http://github.com/donnoman/deprec/tree/webroar

    Sample App Deploy Script using webroar: http://github.com/donnoman/flitten_deploy/commit/76e0ae350d75127fb43e7cc181f0...

    • Tweet
  • Windows XP Caching Nameserver forwarding to Google's Public DNS with support for private wildcard DNS zones

    • 5 Dec 2009
    • 0 Responses
    •  views
    • bind dns forwarding google programming wildcard windows
    • Edit
    • Delete
    • Tags
    • Autopost

    Why on earth would you want to do this?

    For a very specific use case.

    1. You do development of a web application locally that needs a wildcard domain name.  ie: where you want 192.168.1.1 to answer any http request for *.example.com ( example.com, www.example.com, blah.example.com, ridiculously.long.example.com) without specifically configuring each name in a hosts file.
    2. You have previously setup your workstation to use Google's public DNS and don't want to lose the benefits by setting up your own nameserver ( Also applies if you forward to ANY upstream DNS servers, just swap the Google IP's out with the ones you want.)
    3. You are using Windows XP (This should work on Vista). You can apply the same configuration files here for any version of BIND. ( OSX Users should look at a utility called DNSEnabler that provides a dead simple graphical user interface to manipulate BIND on OSX ), but the instructions steps here are specific to Windows.

    Why Google's Public DNS?

    It's faster than your ISP's default DNS which will make browsing faster.  You can also use these instructions for other DNS networks like OpenDNS, or even forward them back to your ISP if you so wish. http://code.google.com/speed/public-dns/

    Why Windows XP?

    Because it's what I have at home. I use OSX and Linux at work where I use DNS Enabler http://cutedgesystems.com/software/DNSEnabler/ on OSX and I've previously posted how to configure BIND on Linux to use Google's Public DNS in an office environment, which pretty much covers all my bases.

    Why BIND?

    It is the defacto DNS implementation, it's well worn and battle tested; and they make a Windows distribution of the software that's freely available.  While I like Microsoft's DNS server, it's only available on their server products, so regrettably it's not an option for Windows XP. 

    There are other Windows based DNS servers, but most of them are commercial and are not as clean, cheap, or as easy as DNSEnabler, so it wasn't worth the time to research any of them. I also have experience configuring BIND so I figured I could just share my configuration and hopefully the 2 other people on the internet that have the same needs as I can benefit from my experience.

    For more information about BIND: https://www.isc.org/software/bind . A great reference about DNS in general as well as BIND is the venerable book "DNS for Rocket Scientists" http://www.zytrax.com/books/dns/

    What are the steps?

    1. Download latest BIND zip for windows: https://www.isc.org/software/bind
    2. Unpack and run: BINDInstall.exe
    3. View: C:\windows\system32\dns\bin\readme1st.txt
    4. Start->Run: C:\windows\system32\dns\bin\rndc-confgen -a
    5. Use Explorer, navigate to c:\windows\system32\dns, right click "etc", properties, security, add, "named", click "full control", OK, OK
    6. Download ftp://ftp.internic.net./domain/named.root and save it to C:\windows\system32\dns\etc\named.root to seed BINDs root hints.
    7. Use Windows Services to start/stop the Named Service
    8. Start -> Right-Click "My Computer", manage , click "Services", Look for "ISC BIND", Right-Click start,stop, or restart.
    9. Start -> Right-Click "My Network Places", properties, Right-Click your active "Local area connection", properties, click "Internet Protocol (TCP/IP)", properties, Use following dns server: 127.0.0.1
    10. For any other system on your network that you want to use this nameserver you would need to use your hosts real IP address for the local network (It may change if your on dhcp, remember to check if you have "internet" problems.) and repeat only step 9 on that system, assuming its Windows.  If it's a Unix platform you would edit /etc/resolv.conf.  If you want to get fancy you can edit dhclient.conf to prevent DHCP from overwriting your custom nameserver selection.
    11. If you customized hosts in your hosts file that will be covered by the wildcard you must remove them: C:\windows\system32\drivers\etc\hosts
    12. If you intend on using the dns server from workstations other than the localhost and you are running any kind of firewall you will need to open up port 53 both UDP and TCP.  For Windows' included firewall Start -> Right-Click "My Network Places", properties, Right-Click your active "Local area connection", Select the "Advanced" tab, settings, Select the "Exceptions" tab, Add Port ( name: DNS, port: 53, type: TCP), repeat last step except for UDP this time.

    C:\windows\system32\dns\etc\named.conf is as follows:

    options {
      // version statement - inhibited for security
      // (avoids hacking any known weaknesses)
      version "get lost";
      // optional - disables all transfers
      // slaves allowed in zone clauses
      allow-transfer {"none";};
      forwarders {8.8.8.8; 8.8.4.4;}; //GOOGLE Public DNS
      directory "C:\WINDOWS\system32\dns\etc";
    };

    view "trusted" {
      match-clients { 192.168.0.0/16; 127.0.0.1; }; // any private class c and localhost
      recursion yes;
      // required zone for recursive queries
      // retrieve from: ftp://ftp.internic.net./domain/named.root
      zone "." {
        type hint;
        file "named.root";
      };
      // basic localhost support
      zone "localhost" in{
        type master;
        file "master.localhost";
      };
      // basic localhost support
      zone "0.0.127.in-addr.arpa" in{
        type master;
        file "localhost.rev";
      };
      // this is the wildcard zone
      zone "dev.example.com" {
        // Don't forward queries for this zone.
        forwarders {};
        type master;
        // The final extension of .txt simply is so that Windows doesn't
        // think the file is executable (.com), and will open the file with your
        // systems designated text editor without any fuss.
        file "dev.example.com.txt";
      };
    };

    view "badguys" {
      match-clients {"any"; }; // all others hosts
      // recursion not supported
      recursion no;
    };

    C:\windows\system32\dns\etc\dev.example.com.txt is as follows:

    $TTL 2d    ; 172800 secs default TTL for zone
    $ORIGIN dev.example.com.
    @             IN      SOA   ns1.dev.example.com. hostmaster.dev.example.com. (
                            2009120500 ; se = serial number
                            12h        ; ref = refresh
                            15m        ; ret = update retry
                            3w         ; ex = expiry
                            3h         ; min = minimum
                            )
                  IN      NS      ns1.dev.example.com.

                                  ; Retrieve the IP for the target of the wildcard
                                  ; Linux: ifconfig
                                  ; Windows: ipconfig
    @             IN      A       192.168.1.1

    www           IN      A       192.168.1.1

                                ; Retrieve the IP for the DNS server
    ns1           IN      A       192.168.1.70

    *             IN      CNAME   www

    C:\windows\system32\dns\etc\localhost.rev is as follows:

     

    $TTL    86400 ;
    ; could use $ORIGIN 0.0.127.IN-ADDR.ARPA.
    @       IN      SOA     localhost. root.localhost.  (
                            1997022700 ; Serial
                            3h      ; Refresh
                            15      ; Retry
                            1w      ; Expire
                            3h )    ; Minimum
            IN      NS      localhost.
    1       IN      PTR     localhost.

    C:\windows\system32\dns\etc\master.localhost is as follows:

     

    $TTL    86400 ; 24 hours could have been written as 24h
    $ORIGIN localhost.
    ; line below = localhost 1D IN SOA localhost root.localhost
    @  1D  IN     SOA @    root (
                      2002022401 ; serial
                      3H ; refresh
                      15 ; retry
                      1w ; expire
                      3h ; minimum
                     )
    @  1D  IN  NS @
       1D  IN  A  127.0.0.1

    To test your new caching dns server for resolving the local wildcard domain: Start->Run: "cmd", then type "nslookup whatever.dev.example.com" at the command prompt.

    (If it doesn't work and you can't browse the web anymore, Look at step 9 and set dns back to "obtain DNS server address automatically", or to 8.8.8.8 and 8.8.4.4, to go direct to Google's DNS. You should also look at your Event Viewer to discover any errors, See step 8, except select the "Event Viewer" instead of "Services", then look at the "Application" log.)

    The command should return:


    Server:  localhost
    Address:  127.0.0.1

    Name:    www.dev.example.com
    Address:  192.168.1.1
    Aliases:  whatever.dev.example.com

    You can use nslookup to check a few more sites like www.disney.com, or www.yahoo.com; to make sure that the forwarding is occuring.

    Your done, enjoy.

    • Tweet
  • tail: can tail multiple files simultaneously, who knew? and other tail tricks.

    • 3 Dec 2009
    • 0 Responses
    •  views
    • programming
    • Edit
    • Delete
    • Tags
    • Autopost

    After using tail for a long time, I've only recently had a need to became familiar with tail's ability to watch multiple files.

    you can easily watch a single file, thats the tail we all know and love.

    tail -f /var/log/syslog

    But I've got another rsyslog directory that concentrates logs from a bunch of different servers with specific naming conventions that I can match by filespec.

    For each cluster there are multiple app, database, loadbalancers and memcache servers. trying to debug a problem, I needed to tail all of the app servers at the same time.

    it's dead simple, particularly if you are in the directory all the files you want to tail reside.

    tail -f *production-app*

    Where it matches any filenames that contain "production-app".

    If I need to watch the mysql servers of my testing cluster

    tail -f *testing-mysql*

    Incidentally you can also tail multiple files without using a filespec

    tail -f /var/log/apache2/access.log /var/log/apache2/error.log

    it tails the files that match the filespec and interleaves the output with markers so that you know which log file you are looking at. Beautiful.


    Other tail tricks:

    combine with grep to watch for your needle before it gets buried in the haystack:

    tail -f /var/log/syslog | grep "my needle"

    combine with grep to exclude a bunch of annoying messages that you don't need.

    tail -f /var/log/syslog | grep -v "annoying message I don't want to see"

    Here's one that I commonly use to cut the cruft out of watching logs on my EC2 instances, it eliminates any lines with Connection OR Kernel in them.

    tail -f /var/log/syslog | grep -v 'Connection\|kernel'

    • Tweet
  • EC2 Instances can now BOOT from EBS, You can now have a non-ephemeral instance on EC2; fantastic!

    • 3 Dec 2009
    • 0 Responses
    •  views
    • programming
    • Edit
    • Delete
    • Tags
    • Autopost
    http://www.allthingsdistributed.com/2009/12/amazon_ec2_boot_from_ebs.html
    • Tweet
  • Using Google's recently announced Public DNS

    • 3 Dec 2009
    • 0 Responses
    •  views
    • programming
    • Edit
    • Delete
    • Tags
    • Autopost
    http://code.google.com/speed/public-dns/

    No forwarders, not previously cached: 259ms.

    ; > DiG 9.3.4-P1 > disney.com
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;disney.com. IN A

    ;; ANSWER SECTION:
    disney.com. 900 IN A 199.181.132.250

    ;; AUTHORITY SECTION:
    disney.com. 86400 IN NS huey.disney.com.
    disney.com. 86400 IN NS huey11.disney.com.

    ;; Query time: 259 msec
    ;; SERVER: 192.168.250.220#53(192.168.250.220)
    ;; WHEN: Thu Dec 3 10:52:19 2009
    ;; MSG SIZE rcvd: 84


    Using Google's Public DNS, not previously cached: 120ms.

    ; > DiG 9.3.4-P1 > disney.com
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 13, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;disney.com. IN A

    ;; ANSWER SECTION:
    disney.com. 900 IN A 199.181.132.250

    ;; AUTHORITY SECTION:
    . 52428 IN NS M.ROOT-SERVERS.NET.
    . 52428 IN NS A.ROOT-SERVERS.NET.
    . 52428 IN NS B.ROOT-SERVERS.NET.
    . 52428 IN NS C.ROOT-SERVERS.NET.
    . 52428 IN NS D.ROOT-SERVERS.NET.
    . 52428 IN NS E.ROOT-SERVERS.NET.
    . 52428 IN NS F.ROOT-SERVERS.NET.
    . 52428 IN NS G.ROOT-SERVERS.NET.
    . 52428 IN NS H.ROOT-SERVERS.NET.
    . 52428 IN NS I.ROOT-SERVERS.NET.
    . 52428 IN NS J.ROOT-SERVERS.NET.
    . 52428 IN NS K.ROOT-SERVERS.NET.
    . 52428 IN NS L.ROOT-SERVERS.NET.

    ;; Query time: 120 msec
    ;; SERVER: 192.168.250.220#53(192.168.250.220)
    ;; WHEN: Thu Dec 3 10:53:45 2009
    ;; MSG SIZE rcvd: 255

    In this trivial and far from scientifically accurate test it appears Google's DNS is considerably faster, and as long as they can continue to maintain this level of performance the use of their servers will be greatly beneficial to our office network.

    We use ISC-dhcpd and BIND on linux servers and configure them on the boxes using vi. There's no pretty Web Based interface on a broadband router here.

    If you have a broadband router, these instructions will not do you any good. Instead, your broadband router probably has a barely-usable web interface, you should RTFM.

    Our DHCP hands out the addresses for two of our local servers that run BIND because we host several domains internally.

    Making the change:

    Assume root status on your name-server

    sudo -i

    Create a time-stamped backup copy of your /etc/named.conf

    cp /etc/named.conf /etc/named.conf.`date +%s`

    Edit the BIND configuration file called named.conf.

    vi /etc/named.conf

    Add the following inside the options {...} section

    forwarders { 8.8.8.8; 8.8.4.4; }; //Google Public DNS

    If you host zones you should exclude them from forwarding

    zone "somedomain.com" IN {
    type master;
    forwarders { }; //don't forward
    file "somedomain.internal.db";
    allow-transfer {
    192.168.0.215;
    };
    notify yes;
    };

    Test the new configurations

    /etc/init.d/named configtest

    Restart Named

    /etc/init.d/named restart

    Rinse and repeat for each of the name-servers that your DHCP server hands out to your clients.

    Get out of root before you screw something else up

    exit

    Let your office enjoy.

    • Tweet
  • Nice write-up on some Virtual Private Server hosts

    • 2 Dec 2009
    • 0 Responses
    •  views
    • programming
    • Edit
    • Delete
    • Tags
    • Autopost
    I've been considering moving several of my shared hosting sites to a single inexpensive VPS. Linode looks like it should be my first candidate.

    http://journal.uggedal.com/vps-performance-comparison

    • Tweet
  • Being a decent citizen

    • 2 Dec 2009
    • 1 Response
    •  views
    • Edit
    • Delete
    • Tags
    • Autopost
    I found and reported a knife on BART today. I'm probably not the only one who saw it, but I am the only one that called and had it picked up. Bart was there and removed it in less than 3 minutes.
    • Tweet
  • About

    Software Developer, Systems Operations, and car-guy.

    78854 Views
  • Archive

    • 2012 (2)
      • April (1)
      • February (1)
    • 2011 (13)
      • November (3)
      • October (1)
      • September (1)
      • July (2)
      • June (1)
      • March (4)
      • January (1)
    • 2010 (27)
      • October (5)
      • August (1)
      • June (1)
      • May (4)
      • April (2)
      • March (4)
      • February (6)
      • January (4)
    • 2009 (7)
      • December (7)

    Get Updates

    Subscribe via RSS
    TwitterFacebook
  • Sites I Like

    • Antique Wreckers on Flikr
    • Hyperbole and a Half
    • The Oatmeal
    • The Big Stump (engines and cars)

    My Other Sites

    • Donnoman’s Garage | Motortopia
    • The Donnoman Daily
    • About.me
    • Blekko
    • Flavors.me
    • Github
    • Sourceforge
    • Flickr
    • Quora
    • Card Cloud
    • Twylah
    • FormSpring
    • DonDon's HubGarage

    Cool Services

    • YouSendIt
    • Pingdom
    • TwitterFeed