Bundle exec - Execute a command in the context of the bundle

This command executes the command, making all gems specified in the Gemfile(5) available to require in Ruby programs.

Essentially, if you would normally have run something like rspec spec/my_spec.rb, and you want to use the gems specified in the Gemfile(5) and installed via bundle install(1), you should run bundle exec rspec spec/my_spec.rb.

Note that bundle exec does not require that an executable is available on your shell's $PATH.

gigaspaces.com - XAP

Application Scaling

• Elastic Application Platform
• Multi-site data replication
• In-memory data grid

Open PaaS

• Cloudify - Open PaaS Stack
  

Ruby Semaphore

Ruby uses the Mutex Class to provide semaphore lock for mutually exclusive access to shared resource.
Example from ruby-doc.org:

Without Semaphore:

count1 = count2 = 0
difference = 0
counter = Thread.new do
  loop do
    count1 += 1
    count2 += 1
  end
end
spy = Thread.new do
  loop do
    difference += (count1 - count2).abs
  end
end
sleep 1
Thread.critical = 1
count1    »    184846
count2    »    184846
difference    »    58126

With Semaphore:


require 'thread'
mutex = Mutex.new

count1 = count2 = 0
difference = 0
counter = Thread.new do
  loop do
    mutex.synchronize do
      count1 += 1
      count2 += 1
    end
  end
end
spy = Thread.new do
  loop do
    mutex.synchronize do
      difference += (count1 - count2).abs
    end
  end
end
sleep 1
mutex.lock
count1    »    21192
count2    »    21192
difference    »    0

Use Mocha to stub a class method and raise an exception

$> gem install mocha

SomeClass.any_instance.stubs(some_method).raises(SomeException)

Extend Ruby core class in Rails application

Follow the ActiveSupport convention, create extension file in lib/core_extensions/[class].rb.
e.g. lib/core_extensions/date.rb

Then, require the file when needed.
e.g. require 'lib/core_extensions/date'

Or put the require statement in any of the config/initializers/ file.

Use Rake to run selected test files

To run specific test file(s) using Rake, one may create a rake file : lib/tasks/individual_test.rake

# Run specific tests for development
namespace :test do
  desc "Run tests related to request."
  task :request => :environment do
    files = %w(
    test/unit/action_request/suspension_request_test.rb
    test/unit/action_request/request_base_test.rb
    test/unit/partials/action_request_test.rb
      )
    files.each do |full_filename|
      sh "ruby -Ilib:test #{full_filename}"
    end
  end
end

Then

$>rake test:request

keywords: rails, rake, test, multiple, file

Rails model use a table name different from its class name

class RequestBase < ActiveRecord::Base
    self.table_name = "requests"
    ...
end

Rails daily usage cheat sheet

Task	Command
Start server	rails s
Start console	rails c
Generate a db migration to add columns	rails generate migration add_column_to_some_table column_name:string... class AddColumnBankAccountNameToRequests < ActiveRecord::Migration def self.up add_column :direct_debit_requests, :bank_account_name, :string end def self.down remove_column :direct_debit_requests, :bank_account_name end end
Generate a db migration to add index	rails generate migration add_index_to_some_table class AddIndexToActionAudits < ActiveRecord::Migration def self.up add_index :action_audits, [:controller, :action, :action_failed] end def self.down remove_index :action_audits, [:controller, :action, :action_failed] end end
Generate new model	rails generate model loan_charge product:references listener_reference:string  fixed_charge:integer min_charge:integer  max_charge:integercharge_percentage:float  class CreateLoanCharges < ActiveRecord::Migration def self.up create_table :loan_charges do |t| t.references :product t.string :listener_reference t.integer :fixed_charge t.integer :min_charge t.integer :max_charge t.float :percentage_change t.timestamps end end def self.down drop_table :loan_charges end end

Configure Rails pages to use different layouts.

• Specify in controller.  By default, the controller will use the same layout (under the view/layouts/ directory) having same name with the controller.  If not found, /layouts/application.html.erb will be used.
• In controller set the :layout value.  details refer to : http://guides.rubyonrails.org/layouts_and_rendering.html

Enable SQL stdout in Rails console

Edit or create .irbrc (or .pryrc if you're using pry) file in user home directory.  Add the following lines:

if defined?(Rails) && !Rails.env.nil?
puts '... ActiveRecord and ActiveResource Logger set to STDOUT'

logger = Logger.new(STDOUT)
ActiveRecord::Base.logger = logger
ActiveResource::Base.logger = logger

end

Then on your rails console, you should see the SQL statements being executed.

Execute SQL statement in Rails

rs = ActiveRecord::Base.connection.execute("select something from sometable")

value = rs[row_id_zero_based]['column_name']

Check if a function is called in Rails unit test

required gem: mocha

some_object.expects(:some_function_name).[expected calls]

where

[expected_calls]

• .once - be call once only
• .never - should never be called.
• .at_most
• ...

Storing array in Rails yml file

Example:

delivery_check_hour_at: [0, 6, 12]

*Note that a space is essential after the comma.

Rails startup sequence

boot.rb -> application.rb -> environment.rb -> /environment/*.rb -> /initializers/*.rb -> routes.rb

Running delayed job in Rails

Example:

Normal run: SomeModule.some_function()

Delayed job run: SomeModule.delay.some_function()

or specify when to run:

SomeModule.delay(:run_at => Time.now + 1.hours).some_function()

Yes, it's that simple!!

SMTP Relay Server

http://sendgrid.com/

Return simple javascript from Rails controller without .js.erb file

In controller:

def some_function

  respond_to do |format|

    format.js { render :js => "alert('hello');" }

  end

end

Create Temp file in Rails

f = Tempfile.new('some_name')

Simple AJAX on Rails

Basic ideas:

• On server side:

1. 1. In routes.rb, setup routings to handle ajax request calls.
   2. In controller, handle ajax request by something like: respond_to do |format| format.js end
   3. In views, create *.js.erb file which has the same name as the corresponding controller function.

• On client side:
  1. In view pages, use tags like <%= button_to , ..., :remote => true, ... %>
  2. Trigger the ajax request.  E.g. by $('#ajax_submit').submit();
  3. Create a javascript like fn_some_callback() to handle callback from ajax response.

Run specify code in Rails after system initialization completed

For development environment,  /config/environments/development.rb

config.after_initialize do

  ...

end

Run Rails console in sandbox

Any changes made will be rollback after exit.

$>rails c --sandbox

Execute external commands in Rails and capture the output

output = %x[./script/delayed_job status]
puts output

Customize a new rake task in rails

Refer to lib/task/*.rake 

Arguments can be passed by using of ENV.

e.g.

desc "create csv of loan ledgers: LEDGERS_START_DATE=2011-07-01 LEDGERS_END_DATE=2011-12-31 rake adhoc:ledgers"
task :ledgers => :environment do
Adhocs.create_csv_of_loan_ledgers(ENV['LEDGERS_START_DATE'], ENV['LEDGERS_END_DATE'])
end

Explicitly trigger an exception notification in Rails

Sometimes you may want to handle an exception nicely and at the same time receive exception notification email.  Here's the way:

begin

...

rescue Exception => e

  ExceptionNotifier::Notifier.background_exception_notification(e).deliver

end

Google Chrome Extensions

http://chromeextensionsdocs.appspot.com/docs.html

Setup SSL on Apache (Mac)

Referece:

http://hints.macworld.com/article.php?story=20041129143420344

http://hints.macworld.com/article.php?story=20080628074917113

Tested on Mac Lion.

STEP 1. CREATING A CERTIFICATE AUTHORITY

Open up Terminal and enter the following commands (don't type the $, that's the prompt):

$ cd ~/Documents

This changes to your Documents folder in your Home directory; next, enter:

$ mkdir certs

This create a new directory called certs; you can name to whatever makes sense to you, although non-spaced names are best.

$ /System/Library/OpenSSL/misc/CA.pl -newca

This runs the CA.pl script that is part of the system to create a new Certificate Authority in the certsdirectory. You will get the following output to the Terminal:

CA certificate filename (or enter to create) Making CA certificate ... Generating a 1024 bit RSA private key ..++++++.................................++++++ writing new private key to './demoCA/private/cakey.pem' Enter PEM pass phrase: (enter a new secure password) Verifying - Enter PEM pass phrase: (reenter the same password) ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave s  ome blank. For some fields there will be a default value, If you enter '.', the field will be left blank. 

As prompted, enter the information prompted for; the more meaningful you make it, the easier it is for people visiting your site to know that they aren't getting a bad connection:

----- Country Name (2 letter code) [AU]: State or Province Name (full name) [Some-State]: Locality Name (eg, city) []: Organization Name (eg, company) [Internet Widgits Pty Ltd]: Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []: sslthis.dyndns.org Email Address []

Once this step is completed, you will have a series of folders inside ~/Documents/certs that make up the necessary structure for a functioning Certificate Authority. Files/directories created so far are:

~/Documents/certs ~/Documents/certs/demoCA ~/Documents/certs/demoCA/cacert.pem ~/Documents/certs/demoCA/certs ~/Documents/certs/demoCA/crl ~/Documents/certs/demoCA/index.txt ~/Documents/certs/demoCA/newcerts ~/Documents/certs/demoCA/private ~/Documents/certs/demoCA/serial 

STEP 2. GENERATE A PRIVATE KEY FOR THE WEBSERVER

The next step will be to generate a private key for your webserver.In the ~/Documents/certs directory, enter the following in Terminal:

$ openssl genrsa -des3 -out webserver.key 1024

This will generate an encrypted, private key called webserver.key; use a meaningful name, no spaces. The output will be:

Generating RSA private key, 1024 bit long modulus ....................................++++++ .....................++++++ e is 65537 (0x10001) Enter pass phrase for webserver.key: (enter a new secure password) Verifying - Enter pass phrase for webserver.key: (reenter the same password) 

Next, you will have generate a non-password protected copy of the key for Apache so that it can start up without errors.

$ openssl rsa -in webserver.key -out webserver.nopass.key

This will generate a non-password protected copy of the private key you just generated.

Enter pass phrase for webserver.key: (enter the secure password created in step 2) writing RSA key

Files generated at this point:

~/Documents/certs/webserver.key ~/Documents/certs/webserver.nopass.key 

3. GENERATE A CERTIFICATE REQUEST

The next step will be to generate a certificate request for your webserver based on the private key generated in step two, in a format that can be signed by the Certificate Authority created in step one. In the ~/Documents/certs directory, enter the following in Terminal (Return key after each entry):

$ openssl req -config /System/Library/OpenSSL/openssl.cnf -new -key webserver.key -out newreq.pem -days 3650 

This will tell the system to generate a new certificate request newreq.pem with the default openssl.confconfiguration file and using webserver.key for a validity period of 10 years.

Enter pass phrase for webserver.key: (enter the secure password created in step 2) You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank. For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]: State or Province Name (full name) [Some-State]: Locality Name (eg, city) []: Organization Name (eg, comp  any) [Internet Widgits Pty Ltd]: Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []:sslthis.dyndns.org Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: leave blank An optional company name []: leave blank 

Files generated at this point

~/Documents/certs/newreq.pem 

STEP 4. SIGNING THE CERTIFICATE REQUEST

The next step will be to sign the certificate request newreq.pem with the Certificate Authority created in step one. In the ~/Documents/certs directory, enter the following in Terminal (Return key after each entry):

$ /System/Library/OpenSSL/misc/CA.pl -signreq 

This will tell the system to sign the 'newreq.pem' file created in step three.

Using configuration from /System/Library/OpenSSL/openssl.cnf Enter pass phrase for ./demoCA/private/cakey.pem: (enter the secure password created in step 1) Check that the request matches the signature Signature ok Certificate Details: Serial Number: 1 (0x1) Validity Not Before: Nov 29 04:00:05 2004 GMT Not After : Nov 27 04:00:05 2014 GMT Subject: countryName = as entered stateOrProvinceName = as entered localityName = as entered organizationName = as entered commonName = sslthis.dyndns.org emailAddress = as entered X509v3 extensions: X509v3 Basic Constrai  nts: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: D8:C4:76:37:6F:8C:FA:8E:62:95:2C:A3:2E:E9:CC:5C:24:E2:5B:DB X509v3 Authority Key Identifier: keyid:DB:12:B4:DB:77:03:D1:64:DA:87:8A:61:79:AA:38:17:E4:7E:6B:ED DirName: emailAddress= serial:00 Certificate is to be certified until Nov 27 04:00:05 2014 GMT (3650 days) Sign the certificate? [y/n]: (type y to confirm) 1 out of 1 certificate requests certified, commit? [y/n] (type y to confirm) Write out database with 1 new entries Data Base Updated Signed certificate is in newcert.pem 

Files generated at this point:

~/Documents/certs/newcert.pem 

After this is done, I moved all the files created (webserver.key, webserver.nopass.key, newreq.pem, newcert.pem) into a new subdirectory, sslthis.dyndns.org, for keeping things nice and neat.

$ ~/Documents/certs/sslthis.dyndns.org

STEP 5. BASIC SSL CONFIGURATION FILE

• Edit /private/etc/apache2/httpd.conf, and uncomment the following line (it's line 473 in my installation):

  Include /private/etc/apache2/extra/httpd-ssl.conf

• Edit /private/etc/apache2/extra/httpd-ssl.conf, and make sure that:
  • SSLCertificateFile points to newcert.pem
  • SSLCertificateKeyFile points to webserver.nopass.key
  • SSLCACertificateFile points to demoCA/cacert.pem
  • SSLCARevocationPath points to demoCA/crl
  Be sure to include the full pathnames for each entry. Optionally, you can edit DocumentRoot to your liking. I point it to /Library/WebServer/Documents-SSL, so I have two roots, one for http and one for https.

============================================

Points to note:

• Use su to perform the setup.
• Define virtual host port 80 on httpd.conf only.  Don't set port 443 on httpd.conf
• Define virtual host port 443 on httpd-ssl.conf.
• Use ">apachectl configtest" to check if the syntax is OK.
• Location of conf files:
  • /private/etc/apache2/httpd.conf
  • /private/etc/apache2/extra/httpd-ssl.conf
    

============================================

The "Passenger" ruby gem: (Install in user login is ok, but has to be su for edit httpd.conf)

> gem install passenger

> passenger-install-apache2-module

Edit /private/etc/apache2/httpd.conf

Add the lines like below as instructed by the output of the above passenger install

   LoadModule passenger_module /Users/xxx/.rvm/gems/ree-1.8.7-2012.02@xxxxxx/gems/passenger-3.0.19/ext/apache2/mod_passenger.so
   PassengerRoot /Users/xxx/.rvm/gems/ree-1.8.7-2012.02@xxxxxx/gems/passenger-3.0.19
   PassengerRuby /Users/xxx/.rvm/wrappers/ree-1.8.7-2012.02@xxxxxx/ruby

Note that the rails will run in "production" environment when it's served by Apache.

============================================

E.g. in httpd-ssl.conf
<VirtualHost _default_:443>

#   General setup for the virtual host
#DocumentRoot "/Library/WebServer/Documents"
DocumentRoot "/Users/yourname/project/project_name/public"
ServerName www.example.com:443
ServerAdmin you@example.com
ErrorLog "/private/var/log/apache2/error_log"
TransferLog "/private/var/log/apache2/access_log"

============================================

Common commands:
$> apachectl stop
$> apachectl graceful
$> apachectl start
$> apachectl configtest

Life is really simple

Life is really simple, but we insist on making it complicated. - Confucius.
世上本无事，庸人自扰之

Choose a job you love

Choose a job you love and you will never have to work a day in your life. - Confucius

知之者不如好之者，好之者不如乐之者。- 孔子 《论语·雍也》

Article on web - How to create very reliable web services

Source: http://amix.dk/blog/post/19709


Over the years I have learnt a lot about creating reliable web services and I have scaled a startup from 0 to millions of users and billions of pages views. I want to share some tips with you that you apply to any language and any website.

The general strategy

I think creating scaleable and reliable services boils down to following:

• Monitor everything: Have a clear picture of what's going on at any time. Have a log of how your past performance has been, have a log of errors, have a log of key metrics in your application, have visualization of every computer in your network, have visualization of response times etc. Using Google Analytics isn't enough, you need much more powerful tools.
• Be proactive: Backups and anticipations of future load should be proactive. If your load is close to maxing out today then don't wait till your web site is down before you begin optimizations. Know everything about your systems so you can take an educated guess of how many more users you can handle. Think about which strategies you can apply if you want to scale 10x, 100x, 1000x of your current load.
  Snippet from Proactive vs. Reactive scaling.
• Be notified when crashes happen: Getting SMS notifications is easy using a service like Pingdom or an open-source library like crash_hound.

The tools that you can use

In my toolset I use a lot of tools and I highly recommend them. There is no need to reinvent the wheel, unless necessary!

Realtime monitoring of anything

I read about statsd and Graphite in Keeping Instagram up with over a million new users in twelve hours.
After I read that blog post I was excited and I used a day to get statsd+Graphite up and running. Graphite is a bit complex to setup, but the investment was great and I must say that it changed my life! This setup lets you track anything in realtime without any performance penalty (since statsd uses non-blocking UDP communication). I track the average response times, number of errors, number of signups, number of sync commands, average response time pr. sync command etc. etc. I track these things across servers and across services.
This is especially useful when you do multiple deployments pr. day as you can quickly see if your latest deployment is introducing errors or performance issues.
Here's one of our Graphite dashboards:

Monitor performance and get notified when something crashes

Pingdom is quite useful as it can track performance from multiple locations in the world. It can also notify you on SMS when your website crashes. Pingdom is also public and lets your users track your uptime and performance (a very useful service for your users).
For more powerful stuff we use crash_hound which lets you program notifications! For example, I get a SMS if a worker queue is not being processed. Most of our notifications are setup using small scripts that are less than 10 lines of code.
How one of our Pingdom monitor pages look like:

Log every error

We have a central logger that logs every error and displays it in a central location. With each error we can see a request's environment. It's a must have tool for any web service since you can track errors and bugs much easier.
Here's how our central logger looks like:

I am sure there are some open-source tools that can do this as well (I did not find any that matched our style so we implemented our own).

SQL and request logger

We have a SQL logger that aggregates queries and shows which are run most times and which are the most expensive. We also have a request logger that shows the same things for requests.
These two tools are essential when you want to know which part of your codebase are worth optimizing.
I open-sourced request logger some time ago (it's very simple and quite hackish!)
Here's a screenshot of how it looks like:

Monitor your servers

Being hosted on Amazon AWS is a privilege since they provide a lot of great monitoring tools (such as CPU usage and alarms if your servers start to use more resources than necessary).
There are some open-source tools available as well. At Plurk we used Cacti and Nagios. Cacti and Nagios are complex and powerful tools, but definitely worth your time since they can give you a much better picture of your servers and services you are running.
Here is how some of Amazon's monitoring tools look like:

I hope you found my tips useful! I might open-source some of my tools such as the central logger or SQL monitor in the near future.
Stay tuned and as always: happy hacking!

Selenium IDE pause

<tr>
  <td>pause</td>
  <td>5000</td>
  <td></td>
</tr>

Selenium IDE input form random value

<tr>
    <td>storeEval</td>
    <td>javascript{Math.floor(Math.random()*11)}</td>
    <td>randomnumber</td>
</tr>
<tr>
    <td>echo</td>
    <td>id=client_short_name</td>
    <td>${randomnumber}</td>
</tr>

Use Selenium to extract web page contents that delays loading

$>gem install selenium-webdriver

require 'rubygems'
require 'selenium-webdriver'

driver = Selenium::WebDriver.for :firefox
driver.manage.timeouts.implicit_wait = 10 # seconds
driver.get "http://food.clickthecity.com/b/sn10f81/jollibee-paco"

phone_number = driver.find_element(:css, 'section#content section#heading div#main-info p span')
puts phone_number.text

driver.quit

Automated browser testing tools

• Watir - http://watir.com/
  
• Selenium - http://seleniumhq.org/
• Sahi - http://sahi.co.in/w/
  

Check Rails version within code

Rails.version

Ruby - Check where does a method defined

Use the "method" call. e.g. some_object.method(:some_method)

or

check its source location:

some_object.method(:some_method).source_location

Attach/add post parameters to form

Add/replace input tag(s) within the form:

function fn_copyFormInputs(sourceFormName, targetFormName, inputNames) {
    for(var i=0; i<inputNames.length; i++){
      $('#' + targetFormName + ' #' + inputNames[i]).remove();
      var value = $('#' + sourceFormName + ' #' + inputNames[i]).val();
      $('<input />').attr('type', 'hidden').attr('name', inputNames[i]).attr('id', inputNames[i]).attr('value', value).appendTo('#' + targetFormName);
    }}