Rails + Merb == Rail3.0 == Merb2.0

I do not think so.

Rails and Merb Merge. I believe this is the worst idea from the software developer point of view.

I have no idea who is the investor for both 37s and EY. But this deal seems it all comes down to $$$. This is a business decision, which is fine. But I just can't understand why it is "such a good thing" for the community like what they advertised. Sigh...

Anyway, I had jobs to be done. I will fork the Merb, it is open source, right?

All that said, good luck with the "merge".

Integrate Ack with Vim

I've been a long time Vim user, but the searching in a project is the missing part in Vim compare to TextMate. I used to switch to a command line using 'grep' or 'rak'(a ruby gem). Finally there is good tip how to integrate it into vim: Ack plugin for Vim

Here is how I do it on my Mac:

1. Install File-Next
2. Install ack
3. Edit your vimrc, add:
set grepprg=ack

4. Create a ack.vim in your .vim/plugin/


Fire up your vim:

:Ack text_to_search_in_current_path

dm 0.9.8 breaks YAML.dump

If you are wondering why your code stop working on dm 0.9.8, this could be one of the reason:
The DM 0.9.8 breaks the after you do any operation on the result Array:


So the workaround could be:
1. Issue only one query to get all you need
or
2. If you can not do 1, then you have to get all the ids, then use (:id.in => ids) to make it working.

Hope this could save you sometime to figure out the very strange error info.

Here is the ticket on light house.

A little tiny bug in Phusion Passenger 2.0.5

If you are using Phusion Passenger like me, I believe you would love it very much. It is such an effort saver. So I've put it on all my machines, even on my dev. , testing and staging machines. But some times I want my Rack apps started in a development mode, so I went through Passenger's document, and added RackEnv environtment variable.



After a restart, I've found that the app was still in production mode. Just played with it for a while and added some debug info, finally I found a workaround, that is, put the RackEnv outside the VirtualHost block.



I am hoping this will save you some time.

P.S. I've submitted a bug report on the issue tracker

Show what branch you are working on in a command prompt

Since I have to juggle between several projects on my typical day job, it would be nice if I can see which branch I am on in a git repository. I figured it out just a single line in my .bashrc:

local GIT='$(git branch &>/dev/null; if [ $? -eq 0 ]; then echo "($(git branch | grep '^*' |sed s/\*\ //))"; fi)'

Then add it to your prompt:

PS1="\u$....${GIT}${NONE}\$ "

Now I get:
~/Projects/spice-workspace/spice2(master)$ git checkout develop
~/Projects/spice-workspace/spice2(develop)$

Very cool!

link_to_action with confirmation

I like Merb, it gives you the best flexibility to extends or hack by yourself. Unlike Rails which gives you every thing that you might need, in merb some of the basic features may need polished. For example, the helper plugin have the delete_button helper method, it deletes the record without get any confirmation. Some times in my app, I want get the user's confirmation first. This is easy, implement your own helper method. There might be a better way to do this, but this works for me:

Git

I was having a problem with my git add -i command, here is the error message:

Can't locate Error.pm in @INC (@INC contains: /opt/local/lib/perl5/site_perl/5.8.8 /System/Lib....

After playing with it for a while, I finally find that when I use mac port to install the git-core, it failed to copy the Error.pm file to the working directory. Here is how to fix it:

$sudo cp /opt/local/lib/perl5/vendor_perl/5.8.8/Error.pm /opt/local/lib/perl5/site_perl/5.8.8/.

I am hoping this could save someone sometime to find the answer.

How to use fixtures with merb

I am using RSpec as the test framework:

spec_helper.rb

# Helper method to adding the fixtures into the tests

def fixtures(*files)

  files.each do |file|

    klass = begin

      Kernel::const_get(Extlib::Inflection.classify(Extlib::Inflection.singularize(file.to_s)))

    rescue

      nil

    end

    entries = YAML::load_file(File.dirname(__FILE__) + "/fixtures/#{file}.yml")

    # do a migrate to create the table to clear the records

    klass.auto_migrate!

    created_objs = {}

    entries.each do |name, entry|

      created_objs[name] = klass.create(entry)

    end

    self.instance_variable_set("@#{file}", created_objs) 

    self.class.class_eval do

      define_method(file) do |key|

        self.instance_variable_get("@#{file}")[key.to_s] 

      end

    end

  end

end

222

Step by step setting up an OS X Leopard server for a Merb/DM/MySQL5.1 application with Phusion Passenger

1. Working env. set up

    a. Update your OS X Leopard

    b. Install Developer pkg

    c. Install MacPort

        - download at http://www.macports.org/

    d. Install git

        $sudo port install git-core

2. Install the latest MySQL server

    http://zhhz.blogspot.com/2008/08/how-to-tune-mysql-server-performance.html

    sudo gem install mysql

3. Install edge Merb/DataMapper

    a. Install Thor

        $ sudo gem install thor

    b. Local installation for a bundled application

        I like to bundle my dependencies with the application, make it independent of my system's gems. This will make it easy to trace with the edge Merb and DM.

        

        - install thor

            $cd your-merb-app

            $curl -L http://merbivore.com/merb.thor > merb.thor

            you can see the tasks by:

            $thor list

        - install edge merb

            $thor merb:tasks:setup

            if you have some errors here, you probably should install rspec first: 

            $sudo gem install rspec

            You will get bin and gems folders

            Install edge merb and dm

            $bin/thor merb:edge --install

            You will get a src folder

            $bin/thor merb:edge:plugins --install

        - install edge dm:

            $bin/thor merb:source:clone git://github.com/sam/dm-core.git

            $bin/thor merb:source:clone git://github.com/sam/dm-more.git

            $bin/thor merb:source:clone git://github.com/sam/do.git

            $bin/thor merb:source:install dm-core

            $bin/thor merb:source:install dm-more

            $bin/thor merb:source:install do/data_objects

            $bin/thor merb:source:install do/do_mysql

            $bin/thor merb:source:install dm-more/merb_datamapper

        - install mongrel

            $bin/thor merb:gems:install mongrel

            You should be able to start your merb app by:

            $bin/merb

4. Install Phusion Passenger

    Details from: http://www.modrails.com/

    a. $gem install passenger

    b. $passenger-install-apache2-module

    c. Modify your httpd.conf

        # c&p from the console of passenger installer:

        LoadModule passenger_module /Library/Ruby/Gems/1.8/gems/passenger-2.0.3/ext/apache2/mod_passenger.so

        PassengerRoot /Library/Ruby/Gems/1.8/gems/passenger-2.0.3

        PassengerRuby /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby

        # config for your app

            <VirtualHost *:80>

                ServerName app.yourdomain.com

                DocumentRoot "abs_path_to_your_merb_app/public"

            </VirtualHost>

        d. Config app for passenger:

            go to your merb app directory:

            $cd your-merb-app

            1. create a tmp folder

                $mkdir tmp

            2. create a file config.ru with following content:

                require 'rubygems'

                if File.directory?(gems_dir = File.join(File.dirname(__FILE__), '.', 'gems'))

                    $BUNDLE = true; Gem.clear_paths; Gem.path.unshift(gems_dir)

                end

                require 'merb-core'

                Merb::Config.setup(:merb_root   => ".",

                    :environment => ENV['RACK_ENV'])

                Merb.environment = Merb::Config[:environment]

                Merb.root = Merb::Config[:merb_root]

Merb::BootLoader.run

                run Merb::Rack::Application.new

            e. Restart your apache server.

5. Modify your spec_helper.rb

    to make your specs working, you have to do the same as the config.ru:

    if File.directory?(gems_dir = File.join(File.dirname(__FILE__), '..', 'gems'))

        $BUNDLE = true; Gem.clear_paths; Gem.path.unshift(gems_dir)

    end

You are ready to go!

UPDATE: The merb part may not good now.

[RUBY] Formating numbers with commas

I want to format the numbers with commas, there maybe a better way to do it, but this one works for me:

  # Return String formated as "123,456.78"

def format_number(number)

  s = "%0.2f" % number

  s =~ /([^\.]*)(\..*)?/

 int, dec = $1, $2

 int.gsub(/(\d)(?=\d{3}+(\.\d*)?$)/, '\1,') + dec

end

How to tune performance of MySQL server

1. Get the latest MySQL Server(5.1)

    a. Download the src from mysql website

   b. Remove the old mysql installation

       - backup you data (mysqldump) first

       - if you compiled and install from src before, go to the src tree

          $sudo make uninstall

       - $sudo launchctl unload -w /Library/LaunchDaemons/com.mysql.mysqld.plist

       - $sudo rm /Library/LaunchDaemons/com.mysql.mysqld.plist

    c. Go to you src tree

       - config (note: I am compiling it with innodb support)

           CC=gcc CFLAGS="-O3 -fno-omit-frame-pointer" CXX=gcc \

CXXFLAGS="-O3 -fno-omit-frame-pointer -felide-constructors \

-fno-exceptions -fno-rtti" \

./configure --prefix=/usr/local/mysql \

--with-extra-charsets=complex --enable-thread-safe-client \

--enable-local-infile --disable-shared --with-innodb

      - $make

      - $sudo make install

    d. Create default database

        $cd /usr/local/mysql

$sudo ./bin/mysql_install_db --user=mysql

$sudo chown -R mysql ./var

    e. Create launchd item

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE< plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>KeepAlive</key>

<true/>

<key>Label</key>

<string>com.mysql.mysqld</string>

<key>Program</key>

<string>/usr/local/mysql/bin/mysqld_safe</string>

<key>RunAtLoad</key>

<true/>

<key>UserName</key>

<string>mysql</string>

<key>WorkingDirectory</key>

<string>/usr/local/mysql</string>

</dict>

</plist>

      and save it to /Library/LaunchDaemons

$sudo chown root /Library/LaunchDaemons/com.mysql.mysqld.plist

   f. start mysql

      $sudo launchctl load -w /Library/LaunchDaemons/com.mysql.mysqld.plist

   g. pump in your backup date

       you probably want to use the innodb engine, so you have to modify your 

         backup data if you were using MyISAM engine:

      $sed -e "s/MyISAM/INNODB/g" -i bak YOUR_DUMPED_DATA_FILE

      then import in with mysql command

NOTE: I would like to give the credit to Dan Benjamin for how to installing mysql

http://hivelogic.com/articles/2007/11/installing-mysql-on-mac-os-x

2. Enable query log for both all and slow

    a. find which my.cnf you are using:

        $mysqld --verbose --help | grep -A 1 'Default options'

         NOTE: you might have to set /usr/local/mysql/bin on your path 

         and symbol link the mysqld from /usr/local/mysql/libexec to 

            /usr/local/mysql/bin

    b. I am using /etc/my.cnf

          log = /var/log/mysql/all-queryies.log

        log-slow-queries = /var/log/mysql/slow-queries.log

long_query_time = 1 # this is second

log-queries-not-using-indexes

log-slow-admin-statements

      Those variables also will put the queries who are not using indexes into the slow log.

   c. change owner

        $mkdir /var/log/mysq

         $touch /var/log/mysql/all-queryies.log

        $touch /var/log/mysql/slow-queries.log

        $chown -R mysql /var/log/mysql

    d. Restart the mysql server

         $mysqladmin shutdown

        then the launchd will restart it automatically

    e. Verify the new config took effectived

              $mysqladmin variables | grep slow

                  | log_slow_queries       | ON               |

                  | slow_launch_time      | 2                   |

                   | slow_query_log          | ON               |

                  | slow_query_log_file | /var/log/mysql/slow-queries.log

              $tail -f /var/log/mysql/slow-queries.log

3. Install monitoring tools

    There are a lot of tools you can choose from. But my favorite one is innotop

     a. Install DBD by CPAN

         $sudo perl -MCPAN -e 'install Bundle::DBD::mysql'

   b. Above step will fail on my Leopard, but you can get ~/.cpan directory

         $cd ~/.cpan/build/DBD-mysql-4.008

    c. Do this to get you make through

         $sudo ln -s /usr/local/mysql/lib /usr/local/mysql/lib/mysql

         $sudo ln -s /usr/local/mysql/include /usr/local/mysql/include/mysql

    d. Make and install

         $perl Makefile.PL

         $sudo make install

    e. Verify

         $innotop

4. Tuning mysql variables and your query statements

Well, this is really something really hardware, application, query statements specific. 

You have to do it by yourself with the tools and logs. But I'v found the following 

variables really matters

       - innodb_buffer_pool_size

      - thread_cache_size

      - key_buffer_size

      - sort_buffer_size

      - query_cache_size

      - read_buffer_size

Do not forget to restart you mysql server after you make any change. I would suggest

you using a version control system to log your my.cnf history.

And for my application and database, adjust following really boots up the performance

       - Adding indexes for the queries who are executing most

        - think before you use SELECT COUNT(*), if you are querying too much data?

Tuning MySQL server

After migrating the data from MaxDB to MySQL 5 [Ver 14.12 Distrib 5.0.62, for redhat-linux-gnu (x86_64) using readline 5.1], I've found our application's performance dropped a little bit. Now as every DB admin might do, I am going to start tuning the MySQL server. Here is my plan:

1. Get current DB status/parameters

2. Develop a benchmark suit before tuning

3. Set up a monitoring system to measure if my tuning improves or hurt the performance

4. Enable slow queries logging

5. Tune SQL queries of my app

6. Version control my.cnf and symbol link it.

7. Maybe, just maybe, order new hardware if necessary.

Migrating from MaxDB(SapDB) to MySQL

I spent some time to migrate data from MaxDB to MySQL today. I googled a bit, but nothing seems working for me. So I have to do it in my way. I've been working with SapDB and MaxDB for a while, I am pretty happy with it. The performance is very good, and the maintenance is easy too. The reason I have to migrate is because I want to give DataMapper(Ruby) + Merb a spin. There is no adapter that are satisfying my need right now, and on the other hand, I am really too busy to write one by myself. Since I am pretty familiar with MySQL, I decided to migrate our MaxDB production database server to MySQL, just for testing for the first phase, we have too many lines of Java code to be migrate too.

Here are the steps, I am hoping this is helpful if you have the same situation like mine.

1. Export MaxDB schema(catalog) and data

a. Schema

/opt/sdb/programs/bin/loadercli -b export_catalog.cmd -d MYPRODUCTIONDB -u MYDBADMIN,MYPASSWD -E 20

the export_catalog.cmd looks like:

EXPORT DB

CATALOG OUTSTREAM 'MYPRODUCTIONDB.ddl'

b. Data

/opt/sdb/programs/bin/loadercli -b export_data.cmd -d MYPRODUCTIONDB -s PRODUCTION_SCHEMA -u USER,PASSWD -E 20 -cs \;

the export_data.cmd looks like:

EXPORT TABLE PRODUCTION_SCHEMA.PURCHASE_ORDERS

DATA OUTSTREAM 'PURCHASE_ORDERS.data';'

EXPORT TABLE PRODUCTION_SCHEMA.PURCHASE_ORDER_LINE_ITEMS

DATA OUTSTREAM 'PURCHASE_ORDER_LINE_ITEMS.data';

...

2. Modify the MaxDB schema to satisfy MySQL grammar, and import into MySQL server.

There are some different between MaxDB and MySQL, you have to modify before you can import the ddl file into MySQL. You have to be very careful about  DEFAULT SERIAL (1)/auto_increment, Integer/INT, fixed/decimal, timestamp...

a. Create db in MySQL

$mysql -h hostname -uroot -pPasswd

>create database MYPRODUCTIONDB character set utf8;

>quit

b. Create schema

$mysql -h hostname -uroot -pPasswd MYPRODUCTIONDB < MYPRODUCTIONDB.ddl

3. Format data before you import into mysql

Because I have too many tables(200+), I write a simple shell script to do that for me

format.sh

#!/bin/bash

for file in *.data; do

 vim -e -s $file < format.vim

done

format.vim

:%s/?/NULL/g

:%s/FALSE/0/g

:%s/TRUE/1/g

w

q

This script will format all the MaxDB data file(*.data) in current folder with the vim. It depends on your data, this could be very time consuming, on my 64-bit RH with 16G memory, it took about 4 hours to get this done. So grab a cup of coffee, chat with your favorite girl in office ...

4. Import data into mysql

Now it is time to get the data imported, for the same reason, I am using a shell script

import.sh

#!/bin/bash

for file in *.data; do

 mysqlimport  -h hostname -uroot -pPasswd --fields-optionally-enclosed-by="\"" --fields-terminated-by=, --local MYPRODUCTIONDB $file

done

That is pretty much all of the steps. If you have some columns in your MaxDB with NULL value, you might have to dump the database from mysql again, substitute them all, then import back again.

4th week

Elaine is growing so fast. She put on some weight, her arms and legs are even stronger than before. Now, she likes playing a bit...

Kicking is her favorite exercise...

Sometimes want to talk a while...
Or just staring the toys...

Can I play, Dad? Please...

The umbilical cord

Today is a big day, finally my baby girl's umbilical cord fell off - it was one week longer than normal. Otherwise she is totally healthy :-)

Sleeping in her crib:

It is 3-weeks now.

More pictures about my baby girl:

Do I know you, Dad?
I want to say...

Back to work

Today is the first day I left her for more than 2 hours since she was born on July-07-2008, I had to be back to work. On my way to office, I kept thinking about her. I had to turn on the radio, otherwise, what I hear was her voice, both crying and the little noisy sound when I was playing with her.

She is such lovely girl, from the first time I saw her, I know I will love her with all my heart forever.

My brand new baby girl and my first blog

I had my blogger account since they just launched their service. For several years, I have no the motivation to start blogging. But, hmm... I changed my mind, this is because:

I've been extremely happy and busy since a few days now. I have the pleasure to be the father of the most beautiful baby girl in the world.