Paperclip Storage S3 Extension

February 18th, 2010 § 0

I recently switched from using attachment_fu to paperclip to store images on S3 in my Rails models. Paperclip is great, as it creates the columns in the model itself without requiring another model (and hence another query to the db). I also use paperclip to store the images uploaded to Amazon’s S3, which works great for me, instead of storing images on the local filesystem.

However, there is one problem that I keep running into time and again. Every few days, I take a dump of the production database and load it in my development environment. Sometimes, I also use the same database in the test environment. The problem, of course, is that unless I duplicate all the files in S3′s storage, I dont see any images, and thus, don’t get nearly the same experience as that of production in either my dev or test environments. Additionally, I do not want to change the bucket configuration in my dev environment to point to the production bucket, as that can create additional problems when I am testing writes and deletes to that bucket.

Wouldn’t it be great that in your dev environment you could still see the images from the production bucket *if* the corresponding image in the dev bucket did not exist? Additionally, if you ever made changes (edits or deletes) to these files, then those changes would be made from the dev bucket and not the production bucket.

So, my friend Shan Bhardwaj, and I ended up writing an extension to the paperclip storage module that does exactly that. You can find the extension at

What this extension does

The paperclip plugin is great to add files to your existing model without requiring an additional model.
The paperclip storage module has a module called S3 which allows you to store your uploaded files in your S3 account.
The buckets can be defined in the configuration YAML file, typically called #{RAILS_ROOT}/config/amazon_s3.yml or
The configuration file looks like

access_key_id: your_access_key
secret_access_key: your_secret_key
bucket: bucket_name_on_S3

access_key_id: your_access_key
secret_access_key: your_secret_key
bucket: bucket_name_on_S3

The problem though is that when you are developing on your machine, or creating a test machine, but with the production environment, then you want to copy the db from your production machine, and play with the data. At this point, you usually want your files (typically images) to be available for your environment. But at the same time, you don’t want to change the files in your production bucket when you change them in your development or test environment.

This is where this extension comes in. The extension allows you to add two more configuration options to your YAML file. So that, the file then looks like

access_key_id: your_access_key
secret_access_key: your_secret_key
bucket: bucket_name_on_S3
bucket_alt: alt_bucket_name
testing: true

access_key_id: your_access_key
secret_access_key: your_secret_key
bucket: bucket_name_on_S3
bucket_alt: alt_bucket_name
testing: false

You can add the additional two options through a capistrano recipe or a chef recipe, based on your deployment mechanism.

Thus, in your production environment, everything will work as before, all reads and writes will occur on the production bucket defined in the configuration bucket:

However, in your test or dev environment, the reads will be done from the alt_bucket_name if it exists (production
bucket), but the writes and deletes will be done from your bucket (dev bucket).


You can get the lib from
Simply add the paperclip_storage_ext.rb file to the #{RAILS_ROOT}/lib/ directory, and require it in the environment.rb file below the Rails Initializer block
require 'lib/paperclip_storage_ext'

If you have questions about this, feel free to drop me a line.

Why PayPal doesn’t quite work for me

February 9th, 2010 § 0

A few days ago, I bought some software for a friend. He lives in Los Angeles, so payment was obviously an issue. I had the software shipped to him directly from the vendor and paid for it myself (it had to be my credit card). We both thought that using PayPal would be super easy, and I would get payment easily enough.

I remembered that many months ago another friend of mine had sent me an invoice and I got charged a fee. Not wanting to pay any fee, I decided to send the invoice to this friend as well, but I still got charged a hefty fee. On a $96.08 transaction, PayPal charged me a fee of $3.09. This is ridiculous. The worst part is that it is not clear at all where and who gets charged a fee. Only recently have PayPal made their fee structure clear and easy to find on their website.

In either case, I don’t think I will be using PayPal anytime soon to ask my friends for money they owe me. A cheque is not too much trouble. It arrives in the mail, and I can deposit it the next time I go to withdraw some cash from my nearby ATM machine.

Rails Extend and Include

February 5th, 2010 § 0

From time to time, I blog about the different things I learn in Ruby on Rails. Recently, I have been engrossed in the metaprogramming aspect of rails. There are some excellent tutorials on the web, but the hierarchy of how methods are called when classes or objects are extended is not discussed thoroughly anywhere. I ran many experiments and found this for myself, so hope that other people will be able to learn from this as well.

Extend vs. Include

“extending” a class with a module adds methods from the module into the class as class methods.
“including” a module into a class adds methods from the module into the class as instance methods.

There is a lot of discussion around this concept, and this is simple to demonstrate. So, if you ‘extend’ a class (say Foo) with a module with method (say bar), then you can call However, if you on include the module into the class, then you can call, where foo is an instance of the class Foo. This is pretty simple to understand.

However, what happens if you already have a method called bar in the original definition of the class. In that case, if you ‘include’ the module into the class, and then call, which method gets called? the bar method in the class definition, or the one defined in the module?

According to the ruby method name resolution algorithm, Ruby searches through the following steps for a name resolution:

  1. As the first step, Ruby checks the eigenclass of o for singleton method named m.
  2. If the method named m is not found in the eigenclass, Ruby search the class of o for an instance method name m.
  3. If the method m is not found in the class, Ruby searches the instance methods of any modules included by the class of o. If there are modules included, they searched in the reverse of the order which they are included.
  4. If no instance method m is found in the class of o or in its modules, then the search moves up the inheritance hierarchy of the super class. Step 2 and 3 are repeated for each class in the inheritance hierarchy until each ancestor class and its included modules have been searched.
  5. If no method named m is found after completing the search, then a method named method_missing is invoked instead. In order to find an appropriate definition of this method, the name resolution algorithm starts over at step 1. The Kernel module provides a default implementation of method_missing, so this second pass of name resolution is guaranteed to succeed.

According to this, the method bar defined in the class will win over the method included in the class. Note that if you extend the class with the module, and there is a method defined in the class as ‘’ will prevail over the one in the module.

This is pretty easy to understand. However, this gets confusing when the instance of the class itself is extended with a module.

  module Mod
     def hello
       "Hello from Mod.\n"

   class Klass
     def hello
       "Hello from Klass.\n"

   k =
   k.hello         #=> "Hello from Klass.\n"
   k.extend(Mod)   #=> #<klass :0x401b3bc8>
   k.hello         #=> "Hello from Mod.\n"

As you can see from the above example, if the object itself is extended with the module, then the method from the module wins over the one defined in the class.

Confusing, isn’t it?

Go ahead, cancel your AT&T contract

February 3rd, 2010 § 0

AT&T sent me a notice detailing that the old ETF is no longer in effect. If you remember, AT&T charges an early termination fee (ETF) of $175 if one cancels the 1-year or 2-year contract with them which was initially used to get a larger subsidy on the cost of the phone. For people who have paid full price for the phone, like me, this was never a good idea and one of the reasons AT&T managed to keep their tentacles hooked into me.

According to the notice, AT&T wireless and Cingular customers who’ve had service any time after January 1, 1998 may be in line for their share of a $18 million cash and “cash benefit” settlement. AT&T also notes that this settlement is for their “old” ETF program, and not for the new pro-rated ETFs that they introduced in 2008.

“We strongly deny any wrongdoing, and no court has found AT&T Mobility committed any wrongdoing regarding these fees. However, we have agreed to settle to avoid the burden and cost of further litigation.

It’s important to note that the litigation involves old early termination fee policies of the old AT&T Wireless and Cingular. In 2008 we introduced a new, more flexible early termination fee policy, in which we pro-rate the ETF if you are a new or renewing wireless customer who enters a one- or two-year service agreement.”

With AT&T’s shoddy coverage in the San Francisco area, I am seriously considering breaking my contract and going towards greener pastures.

How many people LOVE your product?

February 1st, 2010 § 0

Or how many people hate it? I attended a session recently where one of the speakers (Dave McClure) was very passionate about this topic. And with good reason. All of us who have tried making consumer internet ideas a success know that the enemy is not people hating our product, but people just not caring enough to comment, talk about it with others, or in any other way spread the word around.

Most startups rely on some form of word of mouth propagation. Assisted by tools for sharing, it is known as viral, but the basic concept remains the same. In either case, some user has to care enough to click on something and let other people know. People will spread the word around either if they love your product or if they hate it. But if they just like it or don’t care enough, then the product has very little chance of survival. Dave, at the conference, made it more dramatic by suggesting that we think in terms of how many people are willing to ‘fuck’ or ‘kill’ our product.

After coming back home, I realized that it is still very vague. How many people want to fuck my product? I don’t know if anyone wants to fuck Amazon, but there it is, a huge success. Killing the product is easier to understand, but I still fail to imagine how many people want to kill a bad product (maybe a competitor, but the competitor is unlikely to give any press). However, when I started thinking about it, I came up with my own paradigm. In addition to asking how many people LOVE your product, it is also important to ask how many people LIVE your product.

With every consumer internet business, the early adopters are the ones that will make the business. These early adopters do things which sometimes astonish the business owner as to why they are doing it. For example, there are people who have generated thousands of reviews on epinions in the early days. Noone could have know what those people will derive, except some form of satisfaction. The key is that some people should absolutely LIVE your product day in and day out. This means that if you are in a vertical, then go and find those people who live that vertical as a lifestyle. For example, for gigzee, we want to attract the youngsters who live the whole live music/club lifestyle. If even a small number of such people use our product every day and can remember it by name, then the chances of a positive recommendation from them is going to be very high. Additionally, the chances of them recommending to someone else is also very high. These early adopters form the basis for the increasing set of users who will derive the additional benefit from the product.

So, next time when you are designing features to move from 0 to critical mass, ask yourself, how many people will LOVE your product. But don’t neglect how many people will LIVE your product.

For more on how to measure the engagement from these early adopters, read this excellent post by Andrew Chen.

Brainstorming with stupid people

January 30th, 2010 § 0

I hate brainstorming with stupid people. Everyone always says that it’s important to work with smart people. There are more than one reasons for that. Not only do you gain more ideas from other smart people, but they also get your ideas much more quickly. I have seen this time and again everywhere. If you come up with a smart idea, and are explaining it to someone who is not so smart, then they usually end up raising all the obvious objections. After going through the objections, and you answering all the stupid questions, they start forming the same answers that you came up with yourself anyway. Of course, throughout this process, you have now ended up wasting time and effort. Sometimes, these sessions also seem less like brainstorming and more like arguing sessions. The worst part is that you have all the ideas pat down, but are explaining the obvious effects to the less smarter folks in the room. The more stupid the person, the more that person feels that he has contributed more to the idea. Of course, there are some stupid people who just don’t talk. But in my experience, the worst kinds are those who talk before thinking, and raise objections before evaluating an argument.

Many times, these stupid people also draw a person in with absolutely dumb arguments and parallels. It is important for the originator of the idea not to get drawn in these arguments. In any case, the end result is always the same. Everyone in the room comes to the same conclusion that you had started with, except of course, now it’s a collective idea. As time passes, everyone in the group feels that the most important ideas were their own.

I have always maintained that you cannot design deeply in a group discussion. The more people you add to a discussion, the less effective it becomes in finding a resolution. In terms of creative solutions, every additional person reduces effectiveness by half. The best thing to do is to think through the idea, write down the obvious objections and their answers, and then go and present to the group.

Or, better yet, stop brainstorming with stupid people!

Learning how to pitch

January 29th, 2010 § 0

I attended the Twiistup event last couple of days in Los Angeles. It was a great event with lots of interesting people, and some decent companies. As usual, the best part of the conference is meeting with the people, and also seeing where the energy lies in the software space right now. There were 9 companies that were selected to show-off their product, with a tenth slot open for a wildcard entry. The wildcard companies got to present the day before, with an audience vote sending them to the show-off event.

It was really amazing to see that out of the 10 or so wildcard companies, only 2 managed to pitch well. Not really well, but well. The rest 8 were terrible. For all entrepreneurs out there, before you go out to pitch, do yourself a favor, and read the two excellent books by Gene Zelazny – Say it with Presentations, and Say it with Charts.

Not only do these two books make you a better presenter, they will help you structure your content in the most logical way, and present ideas in ways that make sense. And yes, please please please, practice your pitch a few times in front of friends before pitching to others. If that makes you a little uncomfortable, well, that’s the whole idea.

GMail, why can’t I regain full control of my account after being hacked?

December 24th, 2009 § 0

So my gmail account got hacked. Yes, painful, but also very instructive. Firstly, I still do not know exactly how it got hacked. I don’t use any public computer. In fact, I haven’t used any other machine besides my own laptop (Mac) and my iPhone in a very very long time. I don’t sign up for any offers on the internet, and do not install any crap software. Besides, isn’t the Mac supposed to be very safe?

Anyhow, I got hacked and that’s that. The hacker then started sending emails from my account to everyone I have ever communicated with asking for money. The amazing thing about this story is that all the emails sent are very very impersonal, don’t salute the recipient in any way, and are full of grammatical and spelling errors. Yet, the content matter is so sensational (being robbed at gunpoint in some foreign country), that everyone gets worried about my safety. If I received a similar letter, I wouldn’t sit and analyze this, and would fall for it as well. So far, they have preyed on the emotions of their victims through me as the medium.

During the course of trying to get my account back, I ran into some of the issues and got a sneak peak about exactly how these hackers then try to exploit the system, GMail in particular. I had my Yahoo account set up as the secondary email in case of emergencies, or verification. The hacker was quick to change the secondary account first. Gmail has a system of sending verification to a mobile device. This too got changed quickly to some mobile number in Nigeria. During this time, I tried in vain to gain control by asking Gmail to reset my password and send me the password reset code. Gmail only shows that they sent the reset code to, but not the username at the address. So while I was waiting for my password reset code to arrive at my address, the hacker was seeing password reset requests come in to the temporary yahoo address he had set up. I am sure he was laughing at my stupidity and the fact that I sent in multiple requests when the first one failed.

Ok, I was baffled. So I went through the GMail system to report that my account has been compromised. I had to fill in multiple details, including when my account was first started, and the invitation code I used to join (if at all). Obviously, I didnt have any of these, but I made best guesses, and lo and behold, GMail returned my account back to me. I was able to reset my password, and rejoice.

Alas, too soon!

After proclaiming victory, I tried to send a few emails, etc. and it all worked fine. I promptly sent emails to a huge list of people warning them that I had been hacked, and to ignore requests from me for money. There were a few people in the list that I wouldnt have minded getting some money from, but this had to be done.

The hacker, during this time, had very smartly set up a forwarding rule so that he was getting all the emails that I was receiving on my account. This, by itself, is not much. But here comes the most amazing part of how Google engineers missed seeing this as a threat, but these hackers have managed to exploit it. Before I explain what the flaw is, a little diversion into the background.

GMail allows one account to send emails while masquerading as another account. This was designed primarily so that I can have multiple gmail accounts (including Google Apps email accounts like I have a and a account), and still be able to use one primary account and send emails from it for all the different accounts. Great idea, and I love it. All this takes to set up is a simple verification email. So, say you have and If you want to be able to send emails and still show up as, you can go to your settings, set up another email address and this will send a verification email to After clicking on the verification link and entering the code, can now send as If you delete the verification email from the a1 account, there is NOTHING in the settings or account panel of that shows that b2 is still sending emails as a1.

This is exactly what the hacker has done. He has set up another gmail account, and is sending emails on behalf of my gmail account. During this time, he is also receiving the auto-forwarded emails of my account. So even though I have changed my password, and declared victory, he can still receive and send emails just as if he were in full control.

So, step 1, I removed the forwarding rule. Ok, now he cannot get any emails sent to me. Yayyy!
What about his ability to send emails? Turns out that there isn’t any additional verification after the initial verification. What’s more, there is no indication anywhere on my account settings that shows me how many other people can send emails as me. This is terrible. So while I have full control with brand new passwords, the hacker can simply keep sending emails to anyone he likes pretending to be me, ruining my reputation in the process.

GMail – I am not sure how you could have missed this in one of your threat model analyses. But please add an option in account settings where I can control who all can send emails pretending to be me. Meanwhile, the hacker has a field day in sending emails from my account, and can do so as and when he pleases. I am writing a letter to GMail as well so that they can fix this, but if you get any email from me asking for money (personal or not), please don’t wire it to somewhere in Europe. Now, if you want to hand over some cash to me in person, feel free to give me a call!

The real price of gold in the USA

December 2nd, 2009 § 3

So, I have been very interested in buying some gold for investment purposes since the last year. Gold has been rallying like a crazy bull for the past year, with no correction in sight. Partly, this has been because the US dollar has been devaluing, mostly because speculators worldwide are involved in the dollar carry trade, replacing the yen carry trade. However, despite this, the price of gold has been going on up in currencies which are not pegged to the dollar, for example, the euro and the yen. Only the Australian dollar has appreciated in value respective to gold, but this is primarily because the US dollar carry trade is happening with Australian dollars (the Australians are leading the recovery and have had to increase their interest rates).

The chart below shows the price of gold in US dollars. The period before 1967 is not of interest, as the dollar was on the gold standard, thus the price of gold didn’t fluctuate much. In fact, the price of gold remained constant from 1893 to 1918 (world war I).

Price of gold in US dollars

The real question, of course, is whether this is a good time to invest in gold. I am not looking for a short-term play, but my motivation is primarily to protect myself and the family savings from another round of redistribution of wealth that will be brought upon by another round of recession, inflation or devaluation of the currency. So, I decided to compute the buying power of gold, and how that has changed in the last 30 years. For this, I compute the price of 1 lb of flour, needed to make bread. Computing the 1oz gold/1 lb flour ratio, we can see how many lbs of flour would be bought by 1 oz of gold. Luckily, we have this data for the last 30 years, but not much more. Of course, gold had surged in price following the stagflation of the 70s, so 1980 marks the highest point of this ratio, but we seem to be inching our way to those levels now. Thus, the real buying power of gold has gone up significantly, more than anything in the past 3 decades.

gold/flour ratio
Credit: Gold prices from Kitco, and flour prices from

This is a real pickle. On one hand, this signifies that we are going to see inflation, in which case, the price of flour will increase, but so will the price of gold in terms of US dollars. The other side of the coin is that gold is overpriced, as the gold/flour ratio should be relatively constant. However, examining the lows of the ratio in 2001 gives some sense of why gold has appreciated like a raging bull in the past decade. Another fact that is not captured in the graphs is the confidence of the rest of the world in US dollars. As confidence in the dollars increases, people prefer to hold dollars as opposed to gold. A reversal in that sentiment will drive the price of gold higher. This is the reason that we are seeing central banks in India and other emerging countries emptying the gold reserves of the IMF by selling their dollars and hoarding gold instead.

Typical investment strategy says that we should invest at least 10% in gold, to protect from devaluation of the paper currency. What do you think? I would love to know in the comments.

Rails 2.3.4 and acts_as_favorite incompatibility

December 1st, 2009 § 1

Usually, I try to write about information problems, and what we can do to better parse from the myriad of information. I have been involved in gigzee, which is built in Ruby on Rails, and like everyone else, we upgraded to Rails 2.3 a few weeks ago. We also use a customized version of the acts_as_favorite plugin internally to track which artists, gigs and venues people like. So, it was very disconcerting when after the upgrade the acts_as_favorite plugin stopped working for us. Worse, since user favorites is a central theme of our website, it pretty much brought down our entire system.

After lot of looking around the web, and poking around on our servers, we figured out the problem, and are posting our solution so that other people can find it useful. The problem, as we found, is that the acts_as_favorite overloads method_missing to extend the methods for the base class, say the User model. This way, it is able to provide new methods to that class, for example user.favorite_blogs.

Unfortunately, in rails 2.3.4, the file /Library/Ruby/Gems/1.8/gems/activerecord-2.3.4/lib/active_record/associations/association_proxy.rb got changed, to include the highlighted lines. These end up raising a NoMethodError before the overloaded method_missing is called for the @target.

def method_missing(method, *args)
  if load_target
    unless @target.respond_to?(method)
      message = "undefined method `#{method.to_s}' for \"#{@target}\":#{@target.class.to_s}"
      raise NoMethodError, message

    if block_given?
      @target.send(method, *args)  { |*block_args| yield(*block_args) }
      @target.send(method, *args)

Instead of messing with the gem itself, and breaking who-knows-what-else, we decided to simply overload the respond_to? method for the user class.

Our code (in app/models/user.rb):

def respond_to?(method_sym)
  if method_sym.to_s =~"^favorite_(\\w+)")
    return true
  elsif method_sym.to_s =~"^old_favorite_(\\w+)")
    return true
  elsif method_sym.to_s =~"^has_favorite_(\\w+)\\?")
    return true
  elsif method_sym.to_s =~"^has_old_favorite_(\\w+)\\?")
    return true

This essentially goes around the problem by telling ActiveRecord that these methods do exist for the User model, and then the method_missing from the acts_as_favorite plugin is called.

Simple fix, and I hope it can save other people a lot of headache too.

Get Adobe Flash playerPlugin by wordpress themes