Posts from BlogFish...
Posted by Jonathan 33 days ago
After being one week on the road, I'm finally back home again. Last week I presented at RubyFools Copenhagen and Scotland on Rails and therefore traveled a lot.
This was my first time in Copenhagen and it seemed like a very nice city. The conference venue was very nice, a brand new university building with a big hall. My talk about Rails on AWS was well received. I've added some information about the new EC2 features like elastic IPs and availability zones. A video of the presentation should hopefully soon be available. After the sessions I've spend some time chatting with Matz about Ruby/JRuby/Rubinius and his work in Japan.
Next I traveled to Edinburgh for Scotland on Rails. Edinburgh feels like my second home town as I've studied and worked there for a while. The conference was very good organized and had a different Ruby crew there. Most of the RubyFools Copenhagen folks went straight to RubyFools Oslo, so there were many new faces in Edinburgh.
I gave a presentation about Rails Patterns, typical problems of real-life Rails production sites and solutions/patterns. Afterwards I had a couple of nice conversations with other developers and their experiences with similar situations.
Apart from the great conference, I had a chance to spend some time in Edinburgh and catch-up with some people there.
My slides can be found here:
RubyFools Copenhagen: Rails on AWS (PDF)
Scotland on Rails: Rails Patterns (PDF)
RubyFools Copenhagen: Rails on AWS (PDF)
Scotland on Rails: Rails Patterns (PDF)
Posted by Jonathan 47 days ago
Next week I'm going to present Rails Patterns at the Scotland on Rails conference. The idea is to talk about common pitfalls and problems that I've encountered in many of the Rails projects I've been involved with and describe a solution or best practice.
I've already assembled several scenarios like async media processing, handling massive user data or deployment setups. I'm also curious what kind of problems you have encountered in your Rails past so that I can include them.
So if you have any suggestion to a common mistake that you see (like throwing plugins at random problems just because it seems easier) or a useful pattern that you use (like following "vendor everything" or how you handle background jobs), please send them too me. You can post a comment here in the blog, email me or use twitter.
Posted by Jonathan 51 days ago
If you ever got an error like this while testing your code in production
ActionView::TemplateError (private method `chomp' called for nil:NilClass) in
...
<%= stylesheet_link_merged :styles %>
<%= javascript_include_merged :libs %>
then you forgot to create your packed JavaScript and CSS assets:
rake asset:packager:build_all
Posted by Jonathan 60 days ago
Just a small note for people using Capistrano/Webistrano and the remote_cache deployment strategy.
set :deploy_via, :remote_cache
The remote_cache strategy creates a cached-copy directory in your #{deploy_to}/shared base. It then checks out the coe once and in contrast to the default deployment strategy. After the initial checkout subsequent deployments will do a `svn up` and copy the result over to #{deplot_to}/releases/.
Using remote_cache your deployments are usually a bit faster but there is a catch. If you ever change the repository variable, e.g. because you switch to another tag of move the stable branch, your deployments will either fail or do not completely update.
This is due to the fact, that the new deployment does a
$ svn up -rYOUR_REV http://svn.example.com/svn/branches/my_new_branch
on the cached copy. With a different branch or tag than the one the `svn checkout` command was executed with, this will not work. In order to fix it, just delete the cached-copy directory. It will be re-created on the next deployment.
$ rm -rf /path/to/deploy/shared/cached-copy
Posted by Jonathan 85 days ago
The conference season is starting again for me and I wanted to note where I will be/speak during the next couple of weeks.
|
First, there is Ruby Fools Copenhagen (April 1st and 2nd) where I will speak in the Ruby Performance track about Rails on AWS and how to leverage EC2, S3, and SQS in your application. The lineup at Ruby Fools looks really good with speakers like Glenn Vanderburg, Michael Koziarski, Evan Phoenix, Dr. Nic Williams, Dave Thomas, and Matz himself. Unfortunately I will not have too much time in Copenhagen as I have to leave early for Scotland on Rails in Edinburgh.
|
|
I'm really looking forward to be in Edinburgh again. After living, studying, and working there it feels like a second home. At Scotland on Rails (April 4th and 5th) I will talk about Rails Patterns: typical problems and scenarios in Rails applications like asynchronous operations (image processing, calculations, ..), authentication or deployment and common solutions and best practices.
|
In Mai I will be at Linuxtag 2008 in Berlin and hopefully talk about Ruby on Rails Security, but this talk has not been confirmed yet. Further, there is a chance that I will be speaking a the iX Cebit Forum 2008 about our internal Software Development Process and Agile Development.
Posted by Jonathan 97 days ago
|
Design Patterns in Ruby by Russ Olsen is an introduction to Design Patterns. It covers 14 out of the 23 patterns of the GoF Design Patters: Elements of Reusable Object-Oriented Software book and adds three Ruby-related patterns. The book examines each pattern in general, shows how it applies to a dynamic language like Ruby and explains when to use or not use the pattern.
The fact that the book is written in an informal style with lots of examples makes it really easy to read and follow. I really liked the format of the pattern examination and find it an excellent overview of and introduction to Design Patterns. The covered GoF patterns (Template Method, Strategy, Observer, Composite, Iterator, Command, Adapter, Proxy, Decorator, Singleton, Factory Method, Abstract Factory Method, Builder, and Interpreter) are the most important ones and are easy to apply in Ruby. Especially the chapter on Interpreter made a very good job of explaining a widely under-utilized pattern. It even showed how to build a parser and didn't stop at the AST.
|
Apart from those classic patterns, the book introduces Internal Domain-Specific Languages, Meta-Programming, and Convention Over Configuration as newer patterns that are closely related to dynamic languages like Ruby. Those chapters cover nothing new to Rails programmers but are a nice addition to a general Ruby Patterns book. In my opinion the discussion of Meta-Programming could be a bit longer as it only covers class_eval and Ruby has more to offer.
In general, Design Patterns in Ruby is a very good overview of Design Patterns in the modern, dynamic world of Ruby. The book makes sure that the reader understands where a pattern arises and is very good in explaining its usage by example. Further, I cannot praise enough the fact that the author also tells you when NOT to use a pattern and warns about over-usage of inheritance or patterns.
With 338 pages of informal, easy to read examples and explanations, the book is easy to read in two or three afternoons. If you look for an introduction to Design Patterns or want to know how they apply in Ruby, I really recommend Design Patterns in Ruby.
Posted by Jonathan 129 days ago
The slides and a video of my Ruby on Rails Security session are now online. The 24C3 was a lot of fun, unfortunately I couldn't spend all 4 days there.
My talk covered most of the common web application vulnerabilities like Cross Site Scripting and Cross Site Request Forgery, SQL and Code injection, and deployment security and how they apply to Rails. Further Ruby on Rails specific issues like Rails plugin security, JavaScript/Ajax security, and Rails configuration were be examined and best practice solutions were introduced.
The is also a Google video version: Ruby on Rails Security.
Get the slides (PDF - 1.6 MB) or the video (mkv - 95 MB). Other formats are available from the official mirrors or the torrent site.
Posted by Jonathan 186 days ago
I'm just back from today's Web 2.0 Expo sessions and I'm not sure I will attend tomorrow. Many have written about this before, but the creative, social atmosphere is missing due to the conference labyrinth halls. Boy, I'm happy I haven't spend > 1.000 Euros on this. No real food, a lot of product presentations, not enough room for socializing and to many suits for my taste.
Still, I had some nice conversations and met some interesting people.
I did again a session on scaling with Amazon EC2 and S3, the slides can be found here.
This time a also talked a bit about how we use S3 and EC2 to drive our Webmail Portal product, PeritorMail at Peritor.
Also nice the AWS announcement of S3 being available in EU data centers. Now I'm only waiting for EC2 in the EU...
Posted by Jonathan 207 days ago
Webistrano 1.2 is out. You can get it at the Peritor Labs project page.
Version 1.2 brings the following changes and enhancements:
- Fixes a bug in the execution of tasks with sudo
- Highlight recipe syntax with the syntax gem
- Include Capistrano 2.1.0 and git support
- Add experimental support for ssh_keys and ssh_ports as normal configuration parameter (currently only one SSH key is supported)
Although Capistrano 2.1.0 is included, Webistrano will by default allocate a pty for each SSH command as the new Capistrano default of now doing so seems problematic on some hosts.
Further there where some minor fixes and UI enhancements. All available configuration parameter are now documented in the wiki.
Upgrading is done with a `RAILS_ENV=production rake db:migrate`, more information on how to upgrade can be found here.
With version 1.2 out of the door I will focus on bringing authorization to Webistrano so that deployment permissions can be tight to individual users and groups.
Get version 1.2 either from the download page or directly here: webistrano-1.2.zip
Posted by Jonathan 208 days ago
Wow
Amazon EC2 gets two new types of instances, large and extra large EC2 instances. Basically a large instance that has 4 times the capacity (CPU, RAM, HDD) of the old, now default small instance type while the extra large instance type has 8 times the capacity.
Small Instance (default)
1.7 GB memory
1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit)
160 GB instance storage (150 GB plus 10 GB root partition)
32-bit platform
I/O Performance: Moderate
Price: $0.10 per instance hour
Large Instance
7.5 GB memory
4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
850 GB instance storage (2 x 420 GB plus 10 GB root partition)
64-bit platform
I/O Performance: High
Price: $0.40 per instance hour
Extra Large Instance
15 GB memory
8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each)
1,690 GB instance storage (4 x 420 GB plus 10 GB root partition)
64-bit platform
I/O Performance: High
Price: $0.80 per instance hour
The idea is that you specify the instance type in the RunInstances API call. All old tools that do not specify this parameter start a default instance type.
Very nice to see this so fast after the recent S3 SLAs.
If they would now allow to run EC2 instances in Europe there are no excuses left not to run nearly all applications on EC2.
Posted by Jonathan 213 days ago
Capistrano (and thereby Webistrano) is very flexible and there are a lot of variables that can change its default behaviour.
Unfortunately there is no one public list of all parameter to tweak. As the question about all of those comes up every now and then I decided to create a probably incomplete list of all configuration parameter for Webistrano and Capistrano.
The list can be found on the Webistrano project page.
If you have any additions or corrections, please tell me or use the project page directly to submit corrections.
Posted by Jonathan 216 days ago
Finally:
Basically, we commit to 99.9% uptime, measured on a monthly basis. If an S3 call fails (by returning a ServiceUnavailable or InternalError result) this counts against the uptime. If the resulting uptime is less than 99%, you can apply for a service credit of 25% of your total S3 charges for the month. If the uptime is 99% but less than 99.9%, you can apply for a service credit of 10% of your S3 charges.
Very nice to see this shortly after the AWS presentation by Jeff Barr at the Berlin Ruby User Group. But I guess that for many people in big corporate settings this is not enough. But still, it's a start. And it makes selling AWS S3 as part of our solutions easier.
I'n now waiting for a EC2 SLA.
Get the details here.
Posted by Jonathan 226 days ago
A common question about Capistrano tasks is how to capture the output of a task. By default Capistrano will just execute your command and ignore any output.
But sometimes you want to capture and log the output. There is no easy command to do this right now, you have to use the SSH channels:
run "cat /etc/passwd " do |ch, stream, data|
if stream == :err
logger.debug "capured output on STDERR: #{data}"
else # stream == :out
logger.debug "capured output on STDOUT: #{data}"
end
end
You can even send data back depending on the captured output:
run "mysql -u root my_database -p < /tmp/dump.sql" do |ch, stream, data|
if data =~ /password:/
ch.send_data(root_password)
end
end
Using this technique you can store command output in the Webistrano log or just print it if you are using stock Capistrano.
Posted by Jonathan 226 days ago
Webistrano 1.1 is finally out. Webistrano is a Web UI for managing Capistrano deployments. It lets you manage projects and their stages like test, production, and staging with different settings.
It took a little longer because we got some feedback at Rails Conf Europe that we incorporated.
Further Webistrano now has its new home and own Trac/Wiki/Ticketsystem at the Peritor Labs. Peritor Labs is the place where we are going to release inhouse projects like Webistrano under the BSD license. At the moment there is only Webistrano but more will follow.
This means that the Subversion URL also changed. If you tracked Webistrano through svn, please switch to new URL:
svn switch --relocate http://webistrano.rubyforge.org/svn/trunk/ \
http://webistrano.rubyforge.org/svn/trunk http://labs.peritor.com/svn/webistrano/trunk
The major change in version 1.1 is a new shiny UI by the curtsy of Marcus Lamer.
Check out some new screenshots.
Further recipes are now associated to stages and not projects anymore so that different stages of a project can have different recipes. Webistrano now also set two variables inside recipes by default. Those two variables ( :webistrano_project and :webistrano_stage) are set to the project and stage name and can be used to do some decision making inside your recipes.
Upgrading from 1.0 is quite easy, just download the new version, copy your configuration files over and upgrade the database:
RAILS_ENV=production rake db:migrate
The roadmap for the next versions includes some basic monitoring capabilities and some form of permission management that lets you manage which user can deploy which projects and so on. The screencasts have not been updated yet but the content they provide is still valid, the only change is that recipes now belong to stages.
If you have any feedback, please comment here or use the Wiki.
Direct download link: webistrano-1.1.zip or at the Peritor Labs project page.
Posted by Jonathan 231 days ago
If you ever get such a wonderful verbose error message like this while running a Ruby script that uses sockets:
/usr/local/lib/ruby/1.8/drb/drb.rb:837:in `getaddrinfo':
getaddrinfo: Name or service not known (SocketError)
Look for problems in /etc/hosts. In my case I wanted to use the Ferret drb-server and /etc/hosts was completely empty on a fresh EC2 node.
Just add a
127.0.0.1 localhost
And the problem goes away.
Posted by Jonathan 232 days ago
The reason why wrote up my rant about not getting too exited about REST is that I had some fun time paying for my blind usage of REST.
While developing Webistrano I thought that this would be a nice project for playing with all the RESTful stuff that Rails currently offers. So I started to map my resources and thereby only allowing certain HTTP verbs to certain URLs.
This all works nice until reality in the form of IE hunts you down.
Until recently all my Ajax calls used HTTP POST for getting updates from the server. I used POST for so long that I didn't remember why. In Webistrano I use Ajax to periodically get status updates on a running deployment. As getting status updates translates perfectly to HTTP GET on the resource I used this code for it:
# controller
def show
@deployment = @stage.deployments.find(params[:id])
respond_to do |format|
format.html # show.rhtml
format.xml { render :xml => @deployment.to_xml }
format.js { render :partial => 'status' }
end
end
# view _status.rhtml
<% unless @deployment.completed? %>
<% end %>
This worked nicely in Safari and Firefox but Internet Explorer would update the status-div with the whole page. So you got the page-in-a-page effect. I've spend several hours trying to debug from where IE was getting this strange output and why there were no requests to the server. And then I found the answer and remembered why in the past I always used HTTP POST for my Ajax calls.
IE was caching the GET Ajax call.
In order to prevent IE from caching Ajax calls your need to either supply different parameters on each request or switch to POST. Switching to POST is not so easy as Rails will not allow POST requests to the .../deployments/1 resource. So unique parameters on each request it is:
# view _status.rhtml
<% unless @deployment.completed? %>
<% end %>
The alternative would be to define a custom action on the deployment resource that would answer to a HTTP POST but this destroys the whole "one resource URL, different representations" REST thing.
So long for RESTful Web Applications with IE.
Posted by Jonathan 232 days ago
REST is a nice theoretical concept and Rails is pushing hard in this direction. We all heard about RESTful Rails and map.resources but at least in my surrounding its adoption is still lacking.
At the last Berlin Ruby User Group meeting somebody asked if one of us had a running RESTful application and only one out of 40 responded. Then we got into the whole 'what is REST, when is your application RESTful, and is the edit-view part of REST...' discussion.
Yes, it would be wonderful if the world would be full of REST Web Services instead of SOAP Web Services but in the end we are talking about Web Services. If your application does not expose a service, there is no need for hardcore REST. That's why nobody raised their hand, nobody had a service to expose. Not yet, you could argue. And by doing it the RESTful way they would prepare for the day that their application is so successful that they start to think about exposing some resources. In in this case they would be prepared. Just add some responds_to magic and boom, you got your Web Service.
And I answer YAGNI. Do it when you need it, too much preparation is evil.
At the moment some people (especially in the Rails community) see REST as the new Golden Hammer. Just hammer with it once or twice on your application and boom out comes a Web Service for virtually no effort at all. And this is what scares me. Don't get me wrong. I like the idea of REST and definitely prefer it to the WS-* way of doing things. But making your application a Web Service that will serve anything, from JSON to XHTML and XML, just because you can will not solve your problems. It will create new ones because you couple different views and interaction schemes.
Seeing DHH at the RailsConf Europe keynote hacking custom MIME types in order to render a different view for the iPhone didn't make me jump up and down in a jolly manner. It made me thinking if the Rails community pushed too hard to the RESTful side. The iPhone required it's own view because although running a stock Browser it has a different interaction scheme. A different interaction scheme should force you to have a different logic in the controller. The responds_to blocks start to grow in complexity and size.
This is the point where you should apply the idea of de-coupling, the idea of services. Let the iPhone have it's own Web Application. Let than this new application access the common resources that the Desktop Browser application and the iPhone application share. This access can be done through just loading the same models through svn:externals or through a REST Web Service.
The REST concept should de-couple different clients (controller and views) from each other and from and back-end code (models).
Sometimes the responds_to stuff makes sense. If you mostly expose data, then offering JSON and XML besides XHTML makes sense if by doing this you save your clients the transformation or expose the data to more clients that you could not reach before. But it you have a Web Application with a lot of interaction then coupling the Desktop Browser version with the iPhone version, the mobile version, the JSON version, and the XML version does not make any sense to me.
For me REST is about exposing interconnected resources through a unified interface. And yes you should let the client choose the representation. But different clients will handle and interact differently with the resources. So please think before reaching for your Golden Hammer.
Posted by Jonathan 242 days ago
It's been two weeks since the public release of Webitrano . Webistrano was downloded nearly 500 times and I got a lot of feedback. People seem to like the concept and having a tool to manage Capistrano deployments through a Web UI.
A lot of people wanted to tie recipes not to projects but to stages, so Webistrano 1.1 will let you do that. Further 1.1 will interpolate strings in Capistrano variables correctly so that you can refer to other variables inside configuration entries.
Also more recent versions of needle/net-ssh will be included so that Webistrano will not blow up if you have newer versions installed.
Apart from this new functionality/bug-fixing the UI got some love from Marcus Lamer, our UI/Design guy at Peritor. The new UI looks promising:
I hope to get 1.1 out of the door shortly after RailsConfEurope here in Berlin. If you are there and have any feedback, I would love to hear it.
Posted by Jonathan 257 days ago
I'm happy to announce the release of Webistrano - a Web UI for managing Capistrano deployments.
Webistrano is an internal application that I developed at Peritor for the easy handling of Capistrano deployments. I often had situations where our designer updated some images and wanted to update/deploy our Rails project. Capistrano offers a nice command line interface for doing this but this is not the right tool for a designer, especially if you have a complicated multi-stage environment with production, staging and test settings.
Further we wanted to be able to keep track of who deployed what when to which servers.
This is where Webistrano enters the stage.
Webistrano is a Ruby on Rails application that manages projects with their different stages (like production or testing) and leverages Capistrano to handle the deployment part. This way it is very easy to handle multi-stage and multi-client situations and keep an eye on all deployments. It further includes a simple email alerting system so that you get pinged if somebody deploys to the production servers.
We are using Webistrano for quite some time now and it has proven stable for our needs. I am releasing it under the BSD license. The Subversion repository is located at Rubyforge.
On the project page you can find two screencasts that show you how to get Webistrano running and explain some advanced concepts.
| Version 1.0 can be downloaded here: |
 |
| Make sure to watch the screencasts: |
|
I presented Webistrano last month at the Berlin Ruby User Group (then under the name 'Webcap') and the feedback was quite positive. I'd love to hear your feedback and suggestions.
UPDATE:
There was a problem with the Mephisto comment system that resulted in all posted comment being posted to /dev/null. The problem is fixed now, please re-submit your comments.
Posted by Jonathan 258 days ago
After a long period of painful slowdowns due to Typo's caching I finally switched to Mephisto.
Typo created massive amounts of cached paged for each spammer request and this slowed down the whole machine. I had to manually rm -rf the cache directory every now and then.
The switch to Mephisto should be seemless as I migrated all old URLs with Mephisto's redirect handling:
> tail config/environment.rb
# Typo style article URLs
Mephisto::Routing.redirect '/articles/*' => '/$1'
# Typo pages
Mephisto::Routing.redirect '/pages/*' => '/$1'
# Typo feeds
Mephisto::Routing.redirect '/xml/rss' => '/feed/atom.xml'
Mephisto::Routing.redirect '/xml/rss/feed.xml' => '/feed/atom.xml'
Mephisto::Routing.redirect '/xml/rss/comments/feed.xml' => '/feed/all_comments.xml'
Mephisto::Routing.redirect '/xml/rss20/feed.xml' => '/feed/atom.xml'
Mephisto::Routing.redirect '/xml/rss20/comments/feed.xml' => '/feed/all_comments.xml'
Mephisto::Routing.redirect '/xml/atom/feed.xml' => '/feed/atom.xml'
Mephisto::Routing.redirect '/xml/atom/comments/feed.xml' => '/feed/all_comments.xml'
Mephisto::Routing.redirect '/xml/commentrss/feed.xml' => '/feed/all_comments.xml'
I you experience any problems with the URL handling, please contact me.
