Well, this has certainly been a breakout year for Ruby in the St. Louis, Missouri area. Our little Ruby brigade (Stlruby.org[1]) saw its attendance mushroom from an average 7-8/meeting to around 20. So much so that we are seeking a larger space to hold meetings. We have been thankful to Mark Volkman and the staff at OCI [3] for sponsoring our group and providing a place to meet since our second meeting. Thanks Mark and everyone at OCI.
Curt Hibbs kicked off our first meeting at a dinner with Dave Thomas at the Gateway Software Symposium (the "No Fluff, Just Stuff tour,") in March of 2005. I wish I could have been there, but we reprised the dinner this year in March. We got to meet Dave, Bruce Tate, Justin Gehtland, Neil Ford and some other NFJS'ers. These guys are all very young, super brainy and quite funny. I had a blast at the dinner. Pics here[4]
Curt also presented this year at RailsConf in Chicago, a talk titled "Inside Instant Rails," about his ubiquitous InstantRails project for Windows. Curt hasn't sat on his laurels for both the InstantRails and One-Click Ruby projects for Windows. 2006 saw a new book co-authored by Bruce Tate and Curt: Up and Running Ruby on Rails. [5] He also contributed to Ajax Hacks by Bruce Perry [6]. His One-Click Ruby project hit 1 million downloads in November. It is always near the top of the most active projects on Rubyforge. Sliiiiiick!
Our numbers may have grown, but the reason might not be solely due to the explosion of interest in Rails. Occasionally, we've asked our membership what they are doing with Ruby and the answer seems more or less split 50-50 between Ruby and Rails work/interests. St. Louis has a fairly large number of web development organizations and a lot of enterprise shops in the biotech, aerospace and financial industries. Ruby is beginning to make inroads in Java shops and even a few Microsoft shops (of which there are a lot in this area.) Mark has even used Ruby at Scott Air Force Base.
Speaking of Mark and NFJS tour, Mark gave a presentation titled "Ruby Tools" this year. Mark's talks are always well done and very interesting even if you've never heard of the subject. I remember his Distributed Ruby presentation (when we could still all fit in OCI's conference room,) with great appreciation. I went home and built a Rails Wiki using no DB, just the DRb stuff. Mark will return with "Ruby Plays Well With Others" at the 2007 Gateway Software Symposium. The talk will cover JRuby and writing C extensions. A few people from our group attended the talk by Charles Nutter on JRuby at the Gateway Java group. Ruby and Java are certainly two great tastes that taste great together!
Mark and Rob Smith have taught various Ruby and Rails classes at OCI in 2006 with more to come in 2007. Ruby training has seen its biggest increase in 2006. The interest has led two of our members, Mike Sullivan and Jeff Barczewski to form a new training consultancy (Inspired Horizons [7] specializing in Ruby and Rails training. It kicks off in 2007, registration is now open for the first workshops which will cover enterprise stuff with Rails and JRuby. (There is that Java thing again.)
Our membership sure gets around. In addition to Curt, Jeff, Mike and Kyle Cordes attended the RailsConf in Chicago. (I am counting Kyle as a member and sponsor at-large. He currently hosts our web site, and promotes Ruby in his Agile talks. Can't wait for your Lua Users Group to start up, Kyle!) Around town, Curt, Mark, Rob, Jeff and I have given talks to various other users groups. Besides Stlruby, we have presented at St. L Linux (stllug), StLUnix (sluug) St. Charles LUG (stclug) St. Louis Web Developer's (stlweb) and the St. Louis XP users group (xpstl) Several of them and other members also taught at the all day St. Louis Code Camp.[8] David Holsclaw gave a presentation on RoR and Jeff taught a class on Extending Rails featuring MasterView (more on that below.) Cory Foy (a member at-large in outstate Missouri,) taught a program on Ruby for C#/.Net developers comparing the languages and Rails and ASP.Net. I am sure there will be more of this in 2007. Stay tuned.
Speaking of Jeff, you may not be aware of his excellent Rails project MasterView MV is an attribute language for Rails. You can use WYSIWYG editors like DreamWeaver to edit your views as regular .html files and put attributes on tags like a, input and select. MasterView will render them as ERb in .rhtml files on the fly. 2006 saw the release of version 0.3 which featured a new DSL for writing your own customized attributes. Sweet!
Sean Carley has also been very active this year in Ruby projects. He contributed the cat2rb quiz to the weekly Ruby Quiz [9] He also founded the weekly hacking night in St. Louis. Attendance was low, so it is currently on hiatus. There are a lot of monthly IT related events in St. Louis, so much so that nearly every weeknight (and quite a few weekends as well) can entertain us. Balancing that with work and family can be quite tough. Sean has contributed to ruby projects like ParseTree, checkr, and Zentest. He has also been involved in many book and other writing projects. Be sure to check out his blog [10]
Quite a few of us are active in the blogosphere (oh how I loath that particular term. Forgive me.) Besides Sean's check out Curt's writing on O'Reilly's Ruby web log. [11] Curt also has his own Blog featuring a lot of Ruby wisdom [12] Also check out Pat Eyler's (a member on our mailing list, but lives in Utah,) blog [13] Pat also writes for the O'Reilly Ruby blog. Kyle and Cory also have blogs. They are at [14,15] respectively Yippee for self-named blogs!
Some of our members have taken what they have learned and applied it to actual stuff. Steve Molitor and John Hohlen have developed a fantasy golf site using Ruby on Rails. It is just getting off the ground, but they hope to grow the league this year. Check it out at [17]
What about yours truly? Well, I have given 4 presentations on Ruby and Rails at various users groups as I mentioned before. I am presently involved as a partner in a full time Rails consultancy (WDT Solutions [16]). We just finished a rather large Rails+Masterview commercial project for a company in the health care field. I contributed one bug to RSpec and a number of suggestions and feedback to Jeff and Deb on the Masterview team.
Our group has grown so much this past year, that we have decided to actually use a bit of organization. To that end we have formed a steering committee. We meet for about an hour at Friday's before the main meeting. Our first accomplishment is to lay out the presentations for the first few months of 2007. We also migrated the mailing list from Yahoo Groups to Google Groups, which has a much nicer management and user interface.
Well, I am sure I've missed something (or many somethings.) If I didn't cover your book, project, presentation in 2006, please email me and I'll incorporate it. That's it for ruby in St. Louis in 2006. 2007 is shaping up to be another fantastic year in the wide world of Ruby. Prognostication is not usually my forte, but if I had to guess at happenings in '07, I'd say that Curt will become even more famous. Dave and Bruce, et. al. will show up at the NFJS tour and have food with us. More projects will be contributed to, more blog posts written, more presentations will be given, more conferences will be attended. Outside of Japan, Seattle, Portland and Utah, St. Louis will be the place to be for all things Ruby.
Have a Happy New Year!
Ed
[1] StlRuby http://stlruby.org/ruwiki/ruwiki.cgi/
[2] New Google Groups for us: http://groups-beta.google.com/group/stlruby
[3] OCI http://www.ociweb.com/
[4] Pictures from our dinner with dave, Bruce and folks at the Gateway Software conference
http://www.cornetdesign.com/images/DSC00476.jpg (Dave is the one imbibing You can barely see the top of my head)
http://www.cornetdesign.com/images/DSC00477.jpg (Jeff is the one at the left closest to the camera.)
[5] Up and running Ruby on Rails http://www.amazon.com/Ruby-Rails-Running-Bruce-Tate/dp/0596101325/sr=1-1/qid=1167512231/ref=sr_1_1/103-6671475-6469421?ie=UTF8&s=books
[6] Ajax Hacks
http://www.amazon.com/Ajax-Hacks-Bruce-Perry/dp/0596101694/sr=8-1/qid=1167512145/ref=pd_bbs_sr_1/103-6671475-6469421?ie=UTF8&s=books
[7] Inspired Horizons, Ruby Training http://inspiredhorizons.com/
[8] St. Louis Code Camp http://www.stlcodecamp.org
[9] cat2rfb http://rubyquiz.com/quiz77.html
[10] O'Reilly Ruby Weblog http://www.oreillynet.com/ruby/
[11] Curt's Comments http://curthibbs.wordpress.com/
[13] Pat Eyler : On Ruby http://on-ruby.blogspot.com/
[14] Kyls's blog http://kylecordes.com/
[15] Cory's blog http://www.cornetdesign.com/
[16] WDT Solutions, LLC. http://www.wdtsolutions.com
[17] Fantasy Golf Site http://www.rgfgc.com
Saturday, December 30, 2006
Wednesday, August 23, 2006
Testing XML-ish stuff in Rails
Hi fellow Railsians,
Today post is in answer to a question from Chris T on the Rails list. Chris asks:
"Probably dead obvious, but are there any assertions for easing testing
of xml output, both for builder templates (for RSS feed -- something
like a version of assert_tag) and for the new restful stuff."
I've seen this question come up several times in the past. The trouble with assert_tag and assert_no_tag is that they are hooked directly to the response object of your controllers. Which means you can only use them in functional or integration tests. You could fake out a response object, but there is a simpler solution, just recreate them. They are essentially a call to the find method of HTML::Document.
Here is what I did: Add the following to the end your test/test_helper.rb:
I chose to use the actual value as the first parameter rather than the expected value because it allows for nicer looking paramter hash values. To use it, use just like assert_tag and assert_no_tag:
I usually have a default object under test with a single method to output. This allows for a micro-DSL-ish thing to reduce typing overhead:
Today post is in answer to a question from Chris T on the Rails list. Chris asks:
"Probably dead obvious, but are there any assertions for easing testing
of xml output, both for builder templates (for RSS feed -- something
like a version of assert_tag) and for the new restful stuff."
I've seen this question come up several times in the past. The trouble with assert_tag and assert_no_tag is that they are hooked directly to the response object of your controllers. Which means you can only use them in functional or integration tests. You could fake out a response object, but there is a simpler solution, just recreate them. They are essentially a call to the find method of HTML::Document.
Here is what I did: Add the following to the end your test/test_helper.rb:
# Add more helper methods to be used by all tests here...
def assert_xml_tag(xml, conditions)
doc = HTML::Document.new(xml)
assert doc.find(conditions),
"expected tag, but no tag found matching #{conditions.inspect} in:\n#{xml.inspect}"
end
def assert_no_xml_tag(xml, conditions)
doc = HTML::Document.new(xml)
assert !doc.find(conditions),
"expected no tag, but found tag matching #{conditions.inspect} in:\n#{xml.inspect}"
end
I chose to use the actual value as the first parameter rather than the expected value because it allows for nicer looking paramter hash values. To use it, use just like assert_tag and assert_no_tag:
def test_xml_correct
assert_xml_tag "", :tag => 'z',
:parent => {:tag => 'y',
:parent => {:tag => 'x'}}
I usually have a default object under test with a single method to output. This allows for a micro-DSL-ish thing to reduce typing overhead:
...
protected
def assert_render(conditions)
assert_xml_tag @thing.render, conditions
end
Tuesday, August 01, 2006
Simplified user roles in Rails
Usually, the discussion around user authentication and authorization in Rails revolves around an authentication plugin, engine or a login generator. Next, you might get into the types of users and this is where you might see a discussion about Roles, UserRoles and Users. There is plenty of tutorials about this usually describing an example of HABTM or has_many :through. But for simple cases, you can get the effect of user roles without much effort at all, just through normal Rails associations.
In addition to your 'users' table, create tables for each type of role. These can contain attributes specific to that type of user. For instance, you can have a admin type of user with specific flags allowing different types of CRUD access. A business contact user might have additional ways to be contacted (phone extension, cell phone, mail stop, etc.) For each table, add a user_id foreign key. In the models, make sure these all 'belongs_to :user'.
Then in the User model, add a 'has_one :admin' and 'has_one :business_contact' etc. for each associated table. The has_one association adds methods to the model to make it easy to query the relation:
Each user can participate in more than one role. This keeps the user login centralized and the authorization stuff very readable. Adding a new type only requires adding a new table and updating app/models/user.rb to add the has_one associtation.
This simple strategy might not be enough for your needs. It doesn't work if you anticipate adding user roles on the fly to a running system. In this case, you'd probably want to look into a permissions system tied to a HABTM based Role and UserRole model. I've only described a solution for a fixed amount of user types, more or less hard coded into the models.
In addition to your 'users' table, create tables for each type of role. These can contain attributes specific to that type of user. For instance, you can have a admin type of user with specific flags allowing different types of CRUD access. A business contact user might have additional ways to be contacted (phone extension, cell phone, mail stop, etc.) For each table, add a user_id foreign key. In the models, make sure these all 'belongs_to :user'.
Then in the User model, add a 'has_one :admin' and 'has_one :business_contact' etc. for each associated table. The has_one association adds methods to the model to make it easy to query the relation:
user = User.find 1
if user.admin
# do admin stuff
end
...
details.save unless user.business_contact.nil?
Each user can participate in more than one role. This keeps the user login centralized and the authorization stuff very readable. Adding a new type only requires adding a new table and updating app/models/user.rb to add the has_one associtation.
This simple strategy might not be enough for your needs. It doesn't work if you anticipate adding user roles on the fly to a running system. In this case, you'd probably want to look into a permissions system tied to a HABTM based Role and UserRole model. I've only described a solution for a fixed amount of user types, more or less hard coded into the models.
Friday, July 07, 2006
Testing Helpers and Helping Testers
The Rails helper_test plugin [1] is a useful thing that provides you with more tools to better test-cover your code. View helpers that live in app/helpers should be tested in isolation from unit and functional/integration tests. This is true just on the face of it. But additionally, you get to access Rails helper methods which you are usually combining with your own stuff. Hard to do that in normal unit tests, and the setup in integration tests is a little too bloated for my taste.
There are a few tricks that can be used to make this run even smoother. The first trick is to utilize assert_tag to check your HTML generation. assert_tag wants to check the body in the @response object which is expected to be a valid XHTML string. But you are not in the request/response environment of functional/integration tests, so @response is not available. Not to worry, we can just fake it out for now. Create a class with an accessible instance variable that is an instantiated string named body:
In your test method, assign @response = Response.new. Assign the output of your helper method to @response.body. Now assert_tag will work in all it's glory.
The next trick involves testing the image_tag helper. If my helper generates one or more img tags, they can be hard to test with assert_tag. The reason for this is due to the change in Rails to help the various asset tags work better with browser side caches. The helper
image_tag "graphic.gif"
results in something similar:
<img src="/images/graphic.gif?12345678">
where the parameter after the ? is a file based timestamp. This helps the browser to decide to get a new copy of the asset or not. But calling assert_tag :tag => 'img', :attributes => {:src => "/images/graphic.gif?12345678"} is brittle, since if you update the timestamp, (say the next time you get an update from Subversion,) your tests will break. The easy solution to this is to force the ASSET_ID to always be a fixed string. Add this line to config/environments/test.rb:
ENV['RAILS_ASSET_ID']=12345
Now this will be constant in your tests:
[1] http://nubyonrails.topfunky.com/articles/2006/04/07/test-your-helpers
There are a few tricks that can be used to make this run even smoother. The first trick is to utilize assert_tag to check your HTML generation. assert_tag wants to check the body in the @response object which is expected to be a valid XHTML string. But you are not in the request/response environment of functional/integration tests, so @response is not available. Not to worry, we can just fake it out for now. Create a class with an accessible instance variable that is an instantiated string named body:
class Response
attr_writer :body
def initialize
@body = ""
end
def body
"<x>" + @body + "</x>"
end
end
The next trick involves testing the image_tag helper. If my helper generates one or more img tags, they can be hard to test with assert_tag. The reason for this is due to the change in Rails to help the various asset tags work better with browser side caches. The helper
image_tag "graphic.gif"
results in something similar:
<img src="/images/graphic.gif?12345678">
where the parameter after the ? is a file based timestamp. This helps the browser to decide to get a new copy of the asset or not. But calling assert_tag :tag => 'img', :attributes => {:src => "/images/graphic.gif?12345678"} is brittle, since if you update the timestamp, (say the next time you get an update from Subversion,) your tests will break. The easy solution to this is to force the ASSET_ID to always be a fixed string. Add this line to config/environments/test.rb:
ENV['RAILS_ASSET_ID']=12345
Now this will be constant in your tests:
def test_image_asset
@response = Response.new
@response.body = my_helper_that_returns_an_img_tag
assert_tag :tag => "img", :attributes => {:src => "/images/graphic.gif?12345"}
end
[1] http://nubyonrails.topfunky.com/articles/2006/04/07/test-your-helpers
Wednesday, May 03, 2006
StL.rb Hacking Night - Hacking the Ruby Quiz
Well, once again Craig, Sean and I met this past Tuesday for the weekly hacking night of the St. Louis Ruby Brigrade. We decided to work on the current Ruby Quiz which Sean had submitted. It was a lot of fun and a good idea for what to work on if there is no other ideas.
This week's quiz [1] concerned cat2rfb.rb which from the RubyQuiz site he says : "The goal of cat2rafb is to implement a command line utility that will allow you to submit arbitrary text to http://rafb.net/paste/ and get the URL for the text back." You have to understand what rafb.net/paste is all about. This is a site where you can post snippets of code or text then it redirects you to a page where your code is displayed and syntax colorized. But the cool thing is the URL can be shared with others to collaborate.
From their FAQ [2]: "What is a 'nopaste' site?
Sean hadn't actually come up with an answer for his own quiz yet, so that made me suspect he wanted to prime the WHN attendees with some actual cool work to do. In any case, it was a fairly good learning session for me. We actually set up an IRC session on freenode/#stl.rb to share links. Eating our own dogfood, I guess.
The basic idea is becoming familiar with Net::HTTP posts and gets. Sean went for a Ruby Golf approach, while I took the opposite approach. I wanted to provide options and error checking and read from an optional config file. The fields on the nopaste form allow you to choose from different languages, choose a nickname and give it a description. This makes it easier to find in the Recent pastes screen.
The other tricky part is understanding how to follow a redirect from Net::HTTP#post. Turns out Net::HTTPResponse has a bunch of subclasses. So you can do a 'case when' around the response object and then query its 'location' key (it behaves like a hash.)
I also figured out the issues wrt using GetoptLong and ARGF together. You need to process the arguments first, then set a post argument to ARGF.readlines. Also, RDoc::usage is a seriously cool thing. It looks at the comments at the top of your file and processes them into an usage statement and then exits. How can it see the comments in a file that calls it? Very deep Ruby-fu there. But this fits nicely with the DRY rule (comments and usage decared just once.)
So, we finished up right when the meeting was supposed to end. This kind of exploration to solve the quiz was well suited to the amount of time allocated for a hacking night. Look on ruby-talk for mine, Sean's and other answers to this week's quiz.
[1] http://www.rubyquiz.com/quiz77.html
[2] http://rafb.net/paste/faq.html
This week's quiz [1] concerned cat2rfb.rb which from the RubyQuiz site he says : "The goal of cat2rafb is to implement a command line utility that will allow you to submit arbitrary text to http://rafb.net/paste/ and get the URL for the text back." You have to understand what rafb.net/paste is all about. This is a site where you can post snippets of code or text then it redirects you to a page where your code is displayed and syntax colorized. But the cool thing is the URL can be shared with others to collaborate.
From their FAQ [2]: "What is a 'nopaste' site?
A 'nopaste' site allows people to paste chunks of code for others to view. This is useful for situations such as asking for programming help on IRC, where it is frowned upon to paste chunks of code to the channel or to individual. With a nopaste site, the user pastes his or her code to the site and is given a url to provide to others so they can find the code."
What Sean wanted was an easy command line way to paste the code from files, or stdin w/o having to go through the web site. Bonus points if you convert the resulting URL from rafb.net through rubyurl.com which is a Rails equivalent to tinyurl.com that converts long URLs to short ones. (Not that the resulting rafb.net code display URLs are that long anyway.)Sean hadn't actually come up with an answer for his own quiz yet, so that made me suspect he wanted to prime the WHN attendees with some actual cool work to do. In any case, it was a fairly good learning session for me. We actually set up an IRC session on freenode/#stl.rb to share links. Eating our own dogfood, I guess.
The basic idea is becoming familiar with Net::HTTP posts and gets. Sean went for a Ruby Golf approach, while I took the opposite approach. I wanted to provide options and error checking and read from an optional config file. The fields on the nopaste form allow you to choose from different languages, choose a nickname and give it a description. This makes it easier to find in the Recent pastes screen.
The other tricky part is understanding how to follow a redirect from Net::HTTP#post. Turns out Net::HTTPResponse has a bunch of subclasses. So you can do a 'case when' around the response object and then query its 'location' key (it behaves like a hash.)
I also figured out the issues wrt using GetoptLong and ARGF together. You need to process the arguments first, then set a post argument to ARGF.readlines. Also, RDoc::usage is a seriously cool thing. It looks at the comments at the top of your file and processes them into an usage statement and then exits. How can it see the comments in a file that calls it? Very deep Ruby-fu there. But this fits nicely with the DRY rule (comments and usage decared just once.)
So, we finished up right when the meeting was supposed to end. This kind of exploration to solve the quiz was well suited to the amount of time allocated for a hacking night. Look on ruby-talk for mine, Sean's and other answers to this week's quiz.
[1] http://www.rubyquiz.com/quiz77.html
[2] http://rafb.net/paste/faq.html
Monday, May 01, 2006
Ruby w/readline on Ubuntu
Getting Ruby running on Ubuntu 5.10 (Breezy) can be tricky. The issue is Breezy wants Ruby 1.8.3 from apt-get, and that has known problems with Rails 1.x. I usually recommend people install ruby 1.8.4 from source to work with Rails 1.1. But the base install of Ubuntu doesn't have many source build tools and libraries. This shows up in various places. For instance, running 'script/console' barfs complaining about missing the readline require. That is because irb wasn't built with it in the 1.8.4 build tree. Another issue you may have run into is support for rubygems is lacking due to a missing zlib development library. This may have messed up your install of Rails or various plugins.
Before we start read this guide: [1] Be sure to add the extra repositories, if you haven't yet.
Here are a bunch of commands that I executed to get Ruby built.
If you have previously tried to build it, then rm -rf ruby-1.8.4 and tar zxvf ruby-1.8.4.tar.gz; cd ruby-1.8.4; ./configure; make; sudo make install
Retest in your rails app: script/console. Everything should work.
The last install is for openssl development headers and libs. This is a Rake dependency in certain cases. (Not Rails specific.)
[1] http://ubuntuguide.org/
Before we start read this guide: [1] Be sure to add the extra repositories, if you haven't yet.
Here are a bunch of commands that I executed to get Ruby built.
apt-get install build-essential
apt-get install bison byacc gperf
apt-get install zlib1g-dev
apt-get install libreadline5 libreadline5-dev
apt-get install libncurses5 libncurses5-dev temcap-compat
apt-get install libssl-dev
If you have previously tried to build it, then rm -rf ruby-1.8.4 and tar zxvf ruby-1.8.4.tar.gz; cd ruby-1.8.4; ./configure; make; sudo make install
Retest in your rails app: script/console. Everything should work.
The last install is for openssl development headers and libs. This is a Rake dependency in certain cases. (Not Rails specific.)
[1] http://ubuntuguide.org/
Thursday, April 20, 2006
Further adventures along our Trail
More about Migrations.
Here are a few more things I've learned about Rails 1.1 DB Migrations, or I forgot to note in my last post.
If you are using MySQL, then I recommend doing a
The sequence (for legacy DB conversion to Rails Migrations) is:
* This is my rake task. I copy it into lib/tasks (actually I have a generator to do that for me.) The fix_yaml task is needed in my case because I am running MySQL 4.1.x on Ubuntu. The socket is in a different place than Rails thinks and I don't want to be root for my personal DBs I create.
lib/tasks/create_db.rake
Here are a few more things I've learned about Rails 1.1 DB Migrations, or I forgot to note in my last post.
- Make small migrations. It is ok to have very many migrations. Goes with test a little, code a little, repeat, refactor, etc.
- Don't put in the id column for the table. The create_table method does that by default.
- When naming your migrations, only use "pure" CamelCase. E.g. This "AddSSNToClient" doesn't work because ./script/generate migration will creare 003_add_ssn_to_client but the name of the class will be AddSSNToClient, and 'rake migrate' will look for AddSsnToClient.
- Use db/schema.rb to populate your production databases and other development servers. This will still preserve your version # and should allow you to back out to previous versions even on these DB instances.
- Commit any code changes prior to your first migration. Tag this so you know where you came from.
- Use rake db:schema:dump one time when you want to start using migations from a pre-existing SQL DDL generated schema. At this point you will be at version 0. Generate your next model or generate a new migration which will be version 1. If you back out via rake migrate VERSION=0 you will be at the last point before you started using migrations.
- You can port your development environment to a new computer and a new instance of your DB server. Check out your head SVN trunk. I have a Rake task that generates my DBs and sets up config/database.yml*. run that or create at least your development and test dbs. Then run 'rake migrate'. You can backup and restore your old data into your new MySQL instance.
If you are using MySQL, then I recommend doing a
mysqldump -t db_name >db/mydbstruct.sqland commiting that before continuing. You should even make a full backup of your database, which is always a good idea when playing around with DBs.
The sequence (for legacy DB conversion to Rails Migrations) is:
- mysqldump -t mydb >db/mydb_struct.sql
- svn add db/mydb_struct.sql
- svn commit -m "Commit DB Version 000 prior to first migration"
- "svn copy" to tags/DBVersion_000
- ./script/generate migration AddColumnToMytabs
- Edit db/migrate/001_add_column_to_mytabs.rb and add your column(s)
- rake migrate
- rake # to run tests
- svn commit -m "Switched to Rails Migrations. DB Version 001. Added column to Mytabs."
- svn copy to tags/DBVersion_001
- ./script/generate model NewModel
- Edit db/migrate/001_add_columns_to_mytabs.rb
- ... etc.
* This is my rake task. I copy it into lib/tasks (actually I have a generator to do that for me.) The fix_yaml task is needed in my case because I am running MySQL 4.1.x on Ubuntu. The socket is in a different place than Rails thinks and I don't want to be root for my personal DBs I create.
lib/tasks/create_db.rake
require 'yaml'
fn_yaml='config/database.yml'
myuserid='me'
yml=YAML::load(File.open(fn_yaml))
desc 'Creates databases in mysql'
task :create_db => [:fix_yaml] do
databases do |db|
system "echo create database #{yml[db]['database']} | mysql"
end
end
desc 'Drops databases in mysql'
task :drop_db do
databases do |db|
system "echo drop database #{yml[db]['database']} | mysql"
end
end
desc 'Fix database.yml'
task :fix_yaml do
databases do |db|
yml[db]['username'] = me
yml[db]['socket'] = '/var/run/mysqld/mysqld.sock'
end
File.new(fn_yaml, 'w').write yml.to_yaml
end
def databases
%w{development test production}.each do |db|
yield db
end
end
Wednesday, April 19, 2006
Happy Trails
Happy Migrations.
Rails 1.1 has a really cool mechanism for dealing with updates to a database during development: Migrations. Actually, I think migrations predates Rails 1.1, but that is where I first discovered it and it is supposed to be improved. Anyway, migrations are the way you should do all your database development from day 1. It makes the handling database changes much more agile. I tried to wrap my head around migrations from reading online docs, but I was quite unsuited to that task. Thankfully, I attended the 2nd Weekly Hacking Night [1] of StL.rb with Sean Carely and Craig Buchek and I learned what a dunce I had been.
Here are some notes about migrations that I have collected. Others have written eloquently on the mechanics of migrations [2], [3]. I am primarily concerned about the process of migrations. What to do and when to do it.
Lets start with some advice from my little green friend:
Yoda says: If once you start down the migration path, forever will it dominate your destiny, consume you it will.
Lets also assume some starting parameters:
Our Rails app is called trails and it will feature campers, equipment, counselors, hikes, etc. Lets create our databases:
Now lets add our first model: Camper.
Next you need to edit db/migrate/001_create_campers.rb and add columns to the table. Next run
We next create a new model Equipment using the same procress of generate model, edit db/002_create_euipment.rb and rake migrate. Now our development schema_info.version is 2 and we have a new table called equipment.
Next we might decide that each camper is responsible for some pieces of equipment. We need to link our Campers to our Equipment on a foreign key which we forgot to add when we created the model. We do that by generating a new migration:
At this point, we have done a few migations and have a code base that is passing all our tests. Time to check in. Before we do, Sean pointed out that we can use Subversion metadata to keep commits in sync with our DB.
After a few more models have been generated and migrations executed, we might want to revert to a previous database version. Assuming we are at version 7, we just migrate down with :
To summarize:
One last note. If you generate migrations, be sure to name them uniquely. That is because rake will load all the classes in db/migrate in order of version # and Ruby will overrite the first same named class with the last same named class. Use explicit names that reflect what the migration is going to do, e.g. AddClassNameToStudents. This is the ProgramingByIntention [4] principle of XP.
Overall, I have found migrations to fit nicely with the XP test-code-refactor cycle. It opens the database up to being able to be refactored. Subversion is a tool that helps me when I inevitably shoot myself in the foot.
[1] http://sean-carley.blogspot.com/2006/04/stlrb-hacking-nights.html#links
[2] http://wiki.rubyonrails.com/rails/pages/UsingMigrations
[3] http://rails.rubyonrails.org/classes/ActiveRecord/Migration.html
[4] http://c2.com/cgi/wiki?IntentionalProgramming
Rails 1.1 has a really cool mechanism for dealing with updates to a database during development: Migrations. Actually, I think migrations predates Rails 1.1, but that is where I first discovered it and it is supposed to be improved. Anyway, migrations are the way you should do all your database development from day 1. It makes the handling database changes much more agile. I tried to wrap my head around migrations from reading online docs, but I was quite unsuited to that task. Thankfully, I attended the 2nd Weekly Hacking Night [1] of StL.rb with Sean Carely and Craig Buchek and I learned what a dunce I had been.
Here are some notes about migrations that I have collected. Others have written eloquently on the mechanics of migrations [2], [3]. I am primarily concerned about the process of migrations. What to do and when to do it.
Lets start with some advice from my little green friend:
Yoda says: If once you start down the migration path, forever will it dominate your destiny, consume you it will.
Lets also assume some starting parameters:
- You are starting a new project.
- You are using Rails 1.1
- Your DB engine is supported by schema-type Ruby. IOW, you can use db/schema.rb instead of the old schema.sql style. (These are at least MySQL, PostgresSQL and SQLLite, mssql. And some rumored others.) I will be using MySQL here.
- You are using Subversion for version control.
- You are committed to never use SQL to modify your DB structure again.
Our Rails app is called trails and it will feature campers, equipment, counselors, hikes, etc. Lets create our databases:
for d in development test production; do echo create database trails_${d}\;; done | mysql
./script/generate model CamperNote that it created db/migrate and 001_create_campers.rb for us. This is how we are going to create our first table. Note that the version # is 001. As you create new migrations, these will increment, e.g. 002_xxx.rb, 003_yyy.rb, etc. When you run your migrations, rake will load them in that order and the highest versioned file in db/migrate will become the schema version #. More on that later.
exists app/models/
exists test/unit/
exists test/fixtures/
create app/models/camper.rb
create test/unit/camper_test.rb
create test/fixtures/campers.yml
create db/migrate
create db/migrate/001_create_campers.rb
Next you need to edit db/migrate/001_create_campers.rb and add columns to the table. Next run
rake migrateThis will create the table in the development db and create a new table called schema_info with a version column. That version will be set to 1. It will also create a file called db/schema.rb with all our current table settings. Schema.rb will contain the current version of the DB since our last rake migrate, so that is a quick way to discover the current version #. The test framework will use db/schema.rb to create the test database and set its version to 1 as well. Just running
rakewill do that for you, as well as running all your tests.
We next create a new model Equipment using the same procress of generate model, edit db/002_create_euipment.rb and rake migrate. Now our development schema_info.version is 2 and we have a new table called equipment.
Next we might decide that each camper is responsible for some pieces of equipment. We need to link our Campers to our Equipment on a foreign key which we forgot to add when we created the model. We do that by generating a new migration:
./script/generate migration AddCamperIdToEquipmentAdd our column to db/migrate/003_add_camper_id_to_equipment,rb and then run
rake migrateWe are now at version 3. Adding some tests, then runing rake with no options will update our test DB to version 3 as well.
At this point, we have done a few migations and have a code base that is passing all our tests. Time to check in. Before we do, Sean pointed out that we can use Subversion metadata to keep commits in sync with our DB.
svn propset migrate-version 001 .will do that for us. Later, we can svn propget migrate-version . to query it.
svn statusAdd any missing files, then
svn commit -m "Database now at version 003"Whenever I commit following one or more migrations, I like to tag it so it is recorded that the code base is working to a specific DB release.
svn copy -m "DB Version 003" svn+ssh://ip.of.host.box/path/to/repos/trunk svn+ssh://ip.of.host.box/path/to/repos/tags/DBVersion_003
rake migate VERSION=6We can run svn diff to discover what code changes we made since our last tag DBVersion_006 and back those out as well. I generally make it a policy to commit and tag in SVN when I've created a few models (or just one) or I've generated a special migration to add, rename or remove a column.
To summarize:
./script/generate model ModelName
- edit db/migrate/001_create_model_names.rb
rake migrate
rake # runs tests, updates test DB to current schema.
./script/generate model NewModel
- edit db/migrate/002_create_new_models.rb
rake migrate
rake
# add a missing column
./script/generate migration AddColumnToNewModels
- edit db/migrate/003_add_column_to_new_models.rb
rake migrate
rake
# check in
svn propset migrate-version 003 .
svn status # discover unversioned files
svn add db/migrate/001_ , 002 ... etc.
svn commit -m "DB Version 003"
svn copy -m "DB Version 003" svn+ssh://ip.of.host.box/path/to/repos/trunk svn+ssh://ip.of.host.box/path/to/repos/tags/DBVersion_003
...
# revert to a previous version of the DB Current version is 7
rake migate VERSION=6
svn diff svn+ssh://ip.of.svn.box/path/to/repos/tags/DBVersion_006 svn+ssh://ip.of.svn.box/path/to/repos/trunk
# make changes or use svn revert
rake # revert our test DB to version 6 and rerun our tests.
One last note. If you generate migrations, be sure to name them uniquely. That is because rake will load all the classes in db/migrate in order of version # and Ruby will overrite the first same named class with the last same named class. Use explicit names that reflect what the migration is going to do, e.g. AddClassNameToStudents. This is the ProgramingByIntention [4] principle of XP.
Overall, I have found migrations to fit nicely with the XP test-code-refactor cycle. It opens the database up to being able to be refactored. Subversion is a tool that helps me when I inevitably shoot myself in the foot.
[1] http://sean-carley.blogspot.com/2006/04/stlrb-hacking-nights.html#links
[2] http://wiki.rubyonrails.com/rails/pages/UsingMigrations
[3] http://rails.rubyonrails.org/classes/ActiveRecord/Migration.html
[4] http://c2.com/cgi/wiki?IntentionalProgramming
Wednesday, January 11, 2006
What is Green Programming?
This is just an initial post.
Green programming is an attempt to synthesize the principals of environmentally sound policy and the Agile Software set. IOW, just like we should aspire to use renewable energy sources to help the health of the planet, we should also use reusable software elements to create robust, healthy code for our customers. Eco-friendly practices might be thought of as applicable to software devlopment. Just as we are concerned with the various biota and climate of the planet, we should be concerned with the over-all health of the software eco-system. I am specifically concerned with threats to free and open source software from patents and copyright concerns.
Actually, I just needed a title and domain, so I thought I might ride the currently fashionable "Green" movement. Please forgive my impetuousness.
Green programming is an attempt to synthesize the principals of environmentally sound policy and the Agile Software set. IOW, just like we should aspire to use renewable energy sources to help the health of the planet, we should also use reusable software elements to create robust, healthy code for our customers. Eco-friendly practices might be thought of as applicable to software devlopment. Just as we are concerned with the various biota and climate of the planet, we should be concerned with the over-all health of the software eco-system. I am specifically concerned with threats to free and open source software from patents and copyright concerns.
Actually, I just needed a title and domain, so I thought I might ride the currently fashionable "Green" movement. Please forgive my impetuousness.
Subscribe to:
Posts (Atom)