Performance tuning for Grails app in a memory constrained environment

April 20th, 2008

Feedlr has been running quite well for a while, until a couple weeks ago, the server started to choke a lot. And I realized I had to do something to improve the performance. Actually, until then I had barely paid any attention performance-wise - getting the app up and running was the goal. And then with the growth of the app it’s time to do some work for that.

The Problem

The symptom of the performance bottleneck was that from time to time Tomcat would take a long time to respond, resulting in timeouts at the browser clients. This is caused by heavy swapping - apparently the 512MB RAM shared by all the services on the VPS, including Apache (with worker mpm), php, MySQL, and Tomcat, is less than abundant. But before upgrading my VPS, I want to see how far I can go with the current specs.

Attempting to optimize the code

My first attempt was to review my code and try to identify any possible memory leaks and memory inefficient spots. After working with profiling tools (I used an evaluation version of JProfiler, which is very good), and several refractory efforts, there was no obvious improvement. It didn’t look like a memory leak. And the effort trying to squeeze more memory from the code didn’t seem to pay off.

Tuning the JVM

Then I began to focus on tuning the JVM. The sporadic heavy swapping hints at inefficient garbage collections. But why is GC taking so much resources? Using GCViewer, I can see that the throughput can be as low as around 70%. And full GC time constantly reaches several minutes. (SliceHost has sent me a dozen heavy swap usage report already)

After some searching and study, first thing I tried was to decrease the maximum heap size. Yes, decrease it. Previously, I was assigning 384MB to the heap. Because while Tomcat is not running, I have a little less than 440MB free physical memory. Leaving some buffer, 384MB seems a reasonable number. But the excessive swapping and very long full GC time indicates that part of the heap must be slipping into the virtual memory. And virtual memory is the biggest enemy of JVM garbage collection. So I decreased the heap size incrementally, arriving at 270MB. And the GC log shows that the time full GC takes has considerably dropped.

To further improve the situation, I tried different GC policies. First I tried the Concurrent Mark Sweep GC policy, which is supposed to work very well with multi-processor servers and is optimized for response time. Profiling with the CMS options on my local workstation shows good results. And since “cat /proc/cpuinfo” shows that my VPS is equipped with 4 CPU’s, I was expecting to see obvious improvements.

However, to my great disappointment, CMS totally strangled my VPS. Soon after Tomcat starts up, CPU usage became constantly high and the Grails app responds even slower! I’m not exactly sure why, but it looks like the number of CPU’s shown at your VPS doesn’t mean it has the comparable concurrent processing power.

Then I gave up the CMS policy and tried the other GC policy suitable for servers, the Parallel policy. This turned out to work quite well on the VPS, together with “-XX:+UseParallelOldGC” which is available since Java 5 update 6 and enhanced in Java 6 which I’m running my app on.

The Results

I experimented with a few other JVM options, and finally achieved a throughput of above 98%. Full GC now usually takes around 1 second, occasionally maxing at over 10 seconds. I think there are some hard limitations to the server, but the results are already good enough for now. Here are the current ones which have achieved reasonable performance for my Grails app. These may not be the perfect setup since I haven’t tried the combinations exhaustively.

-server -Xmx270m -Xms270m -XX:MaxPermSize=80m -Xverify:none -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+UseAdaptiveSizePolicy -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -XX:+AggressiveOpts

References

Below are some useful articles I’ve come across while tuning the JVM.

Further thoughts

I really love Grails for what it is: remarkably agile development with real joy. But as the person running the app, I have to deal with various issues of the stack that Grails sits on, which sometimes frustrates me.
On one hand this robust stack really grants Grails an advantage in the enterprise market. But on the other hand, this is no good news for the creators of hundreds and thousands of small web apps, pet projects, interesting weekend hacks and alike. The most obvious obstacle is Java hosting. It’s 2008 already, and there are still simply _no_ easily affordable and good enough hosting services for the Java stack! The entry cost is too high. With innovative services like Heroku and Googel App Engine emerging, the gap is even increasing. The barrier to actually making something useful on the web is being dramatically lowered. But yet, the Java stack is left out in the cold.

While this is a problem, it’s also an incredible opportunity. With Grails emerging, the Java stereotype can be changed. People can build agile web apps on top of the proven stack, which is the core competency of Grails as I see it. I want to build things with Grails. But I don’t want to worry about how to host my app. Now, who’s gonna give us the ultimate service to grow little ideas into reality?

Feedlr新功能:预览消息样例

April 8th, 2008

在Feedlr建立一个新Feed的时候,现在可以直接预览在当前设置下播发到微博客帐号上的消息格式样例了!

image

在新建Feed向导的Feed信息设置页面上,如果在Feed地址里填上了正确的地址,点击“预览消息样例”按钮,就能看到以最新一条Feed内容为例子,机器人将会播发的消息格式。

如果没有正确的Feed地址,那么样例会用“标题”,“摘要”和“链接”来代表Feed的内容部分。在修改了Feed地址以及“更多选项”里面的消息前缀,播发标题、摘要、和链接的选项后,再点预览按钮就能更新消息格式。

消息预览是为了让用户更方便的理解并选择合适的选项,但并不一定100%和最终播发的消息完全一致。比如Feed条目的链接默认会转换成tinyurl以后再播发出去,但也会根据Bot选项来决定是否使用原来的完整地址。

希望这项小功能给各位带来一些实用的帮助。

Feedlr十八般用法之:消息前缀的妙用

April 6th, 2008

不知道大家有没有注意到,在新建一个Feed广播的时候,可以在Feed信息的“更多选项”里填上给每条消息加上的前缀。

image

这个功能有什么用呢?最简单的用法,可以用博客的标题,Feed的内容关键字,等等作为前缀,这样关注者收到就知道消息是从哪里来,关于什么的了。但今天我想介绍的是几种更有趣的用法。

@通知某人

如果设定了前缀“@用户名”,那么发送的每条消息都能通过@功能让某人(用户名)收到。目前Twitter,叽歪,饭否都支持@功能。

有什么用呢?比如你向自己的微博客帐号发送消息,如果不加@前缀,那么新消息别人都能收到而你自己收不到。如果加上前缀“@你自己的用户名”,那么你和你的帐号的关注者都能收到新消息了。

另外,@功能可以让不关注这个帐号的人也能收到消息通知。

把消息做为私信发送给某人

想到了@的用法,可能你也就想到了d的用法。在Twitter和叽歪,支持用格式“d 用户名 消息内容”来直接发送私信给某人。如果给你的feed消息加上前缀“d 用户名”,那么每条feed消息都会直接做为私信发送给这个用户。

这样的话,你的feed机器人就成为某人(大多是你自己吧)的专用了,feed的消息不会被微博客上的其他人看到。

用前缀定义track关键字

Twitter和叽歪都支持关键字track功能,但是各自的使用方式有所不同。

在Twitter,用户可以通过GTalk或者手机发送“track 关键字”来自定义关键字跟踪。这样,只要有人在Twitter发表了包含关键字的消息,用户就能收到这条消息。

而在叽歪,如果一条消息带有“[关键字]”的内容,那么方括号里的关键字就自动成为一条线索。用户可以通过访问“http://jiwai.de/t/关键字”的地址来看到所有带有这个关键字的消息。在关键字页面上,用户可以订阅feed或者关注这个关键字。在IM和手机上,也可以发送“on [关键字]”来直接关注。

那么,如果你用feedlr广播到Twitter或者叽歪的话,你自然可以给你的feed加上相应的关键字前缀,就能保证关注这个关键字的用户都能收到feed的更新了,而不论用户是不是关注了你的机器人帐号。

Feedlr十八般用法之:让国内用户读到Feedburner烧制地址的Feed

March 30th, 2008

国内喜欢订阅Feed的朋友肯定都有一个头痛的问题,由于Feedburner的撞墙,一些用Feedburner烧制的Feed在阅读器里的经过Feedburner重定向的链接就不能直接打开读全文了。

比如我自己在Google Reader里80%经常读的Feed都是用Feedburner的。如果Feed内容包含了全文,可以直接在Reader里看,那么没问题。但要是想点Feed文章的链接去网站看全文,就撞墙了。而且讨厌的是,我要看的网站本身往往都不会撞墙,只是因为要经过Feedburner重定向才出的问题。

所以,我自己非常喜欢Feedlr的“把FeedBurner地址转成原地址”这个功能。这个小小的功能,能在把Feed内容播发到迷你博客之前,把Feedburner重定向地址转换成网站的原地址,再播发出去。这样一来,迷你博客的读者就能直接通过接收到的链接读到Feed全文网页了,不会再撞墙了!

进一步的用法,可以把Feedburner的feed播发到迷你博客上,然后在阅读器里直接订阅迷你博客帐号本身的Feed,这样在阅读器里就能看到原文的文章链接,绕过Feedburner重定向了。

使用Feedlr的这个实用功能非常简单,只要在新建向导的第二页,“输入Bot资料”里,选上这个选项就行了:image

Feedlr十八般用法之:把自己的博客同步到叽歪,饭否,和Twitter

March 28th, 2008

一眨眼3月初起公开测试的Feedlr马上要满月了!feedlr日志之前的文章基本都是记录feedlr开发中的一些经验和技巧,但是还没有中文介绍。从今天起这里会有一系列feedlr使用技巧的中文文章,希望通过这些介绍能让feedlr为各位朋友提供更多的用处,成为各位在网络上的好工具好朋友:)

首先,feedlr是什么呢?一句话来说,feedlr是一个能把任何网上的feed内容同步播发到迷你博客上的在线工具。

什么是feed呢?

大家注意到没有,你在浏览网络的时候,现在能越来越多的发现这样一个图标 每当你看到这个图标的时候,你就知道,现在浏览的这个网站有feed可以提供。feed其实是多种xml格式文件的统称,feed的用处是把网站的内容以xml格式抽取出来,这样其他程序就能方便的得到网站的内容,做出各种有用的应用出来了。

如果你已经是Google Reader,抓虾,鲜果等等Feed阅读器的用户,那么应该对Feed已经很熟悉了吧!

什么是迷你博客呢?

迷你博客的概念是由国外的Twitter建立的,简单的说就是能方便的发布一两句话到网上,让朋友能马上知道你在做什么的一种服务。国内也有不少迷你博客服务,最常用的有叽歪饭否

迷你博客虽然也叫做博客,但其实用处和传统的博客有很大不同,更多的其实是一种实时沟通平台。如果你注册了一个迷你博客,只是用来在网页上发表几句话的话,那么你大约只用到了10%的功能。而当你把自己的各种IM帐号,手机,等等联系工具都绑定到迷你博客上以后,你会发觉你和朋友之间的沟通方式彻底改变了。

那么,为什么要把自己的博客同步播发到迷你博客上呢?

有了自己的博客以后,最需要做的事情是什么?就是让更多的朋友知道你的博客,访问你的博客。这样做的方法也有不少。但是,进一步来说,怎么能让你的读者第一时间读到你博客的更新呢?读者阅读的方式一直都是“拉”的方式,也就是说,读者需要时不时地想起来在网上看一下你的博客有没有更新。如果一直忘了,或者上不了网,那就读不到你的最新大作了。

而迷你博客对你的关注者来说是一种可选的“推”的形式。只要关注者打开了通知,你的更新就能马上被送到关注者那里。可以通过电子邮件,IM,甚至离线的时候能通过短信发送到读者的手机。

那么如果能把博客文章自动发布到迷你博客上,读者不就能随时随地第一时间受到博客的更新了吗?这就是feedlr的基本功能!

怎么让feedlr把我的博客同步播发到我的迷你博客呢?

过程很简单,总结起来只需要3步:

  1. 登录或者注册以后,点击页面顶上的“新建”选项进入向导: image
  2. 输入Feed的地址,然后按“验证Feed”按钮:image
    Feedlr会很快的连接一下Feed来验证你填的地址是不是ok,验证没问题的话feedlr会帮你把标题和描述自动用Feed的信息填上。其余选项都是可选的,可以直接按image 继续
  3. 在第二页也是最后一页上,填好用于bot(机器人)的迷你博客帐号,选择一个迷你博客类型,然后同样按image让feedlr检查一下帐号确认没问题,最后按下提交按钮,你的feed机器人就搞定了!

问题1:”bot”,“机器人”到底指的是什么?

bot就是机器人的意思,特指一些网络机器人。在feedlr,一个bot帐号指的就是你用来播发Feed内容的迷你博客帐号。它本身就是一个普通的迷你博客帐号,但在登记到feedlr以后,就会自动播发feed更新,所以就可以称作一个迷你博客机器人(bot)了。

问题2:我该用什么帐号做bot呢?

由于目前迷你博客普遍的帐号验证方式,feedlr需要知道bot帐号的用户名和密码才能自动播发消息过去。feedlr在服务器上尽可能保证帐号的安全,但还是建议大家为bot专门安排一个迷你博客帐号,不要用自己的主帐号,给机器人一个“马甲”。不过当然,任何帐号都是可以的。

问题3:我填的帐号为什么总是验证出错?

有几种可能。一是你可能填错了用户名。一般来说,一个用户在迷你博客上有email地址,用户名,和完整姓名或者screen name这3种标识,在feedlr上填的应该是用户名,而不是其他两种。可以到迷你博客帐号设置里查一下就清楚了。

另外是中文用户名。目前为止feedlr还不能验证中文用户名。这个问题会尽快解决的,但是现在还是请用英文用户名作为bot帐号。

更新:feedlr可以验证中文用户名的帐号了,叽歪和饭否都支持中文用户名,但是Twitter本身是不支持中文名的。 

好了,现在你的第一个Feed广播机器人已经诞生,开始广播啦!

有想法就有可能!

Feedlr的功能尽可能做到通用,提供实用的选项,鼓励DIY,让大家自己灵活使用,来做出许多实用的机器人。同步自己的博客只是最基本的用法,只要你想的到,feedlr可以有更多有趣的用法!

今天先举一个feedlr的朋友在“可能吧”提出的利用feedlr让手机免费接受天气预报短信的文章,就是一个有趣而且实用的用法。已经有不少朋友在feedlr建了天气预报机器人,可以从这里这里看到。当然,除了文章介绍的饭否,同样可以在叽歪和Twitter上做同样的事情,而且据我经验叽歪的短信支持最好,可能更适合收天气预报。

叽歪还在官方博客上介绍了Feedlr,提到一些适合叽歪的特定功能。

好了,今天就介绍这些吧,以后还会介绍更多新鲜有趣的用法。大家如果想给我留言的话,不管是碰到问题还是建议意见,可以直接到这里发言噢!我马上就能收到你的消息了:)

How to create an interactive Twitter bot with Grails in 10 minutes

March 18th, 2008

A while ago I read this interesting post from Glen Smith about connecting to Google Talk with Grails. And I thought, “Wow, that’s really easy!” With merely a dozen lines of code you can already send messages via XMPP to Google Talk clients! And the other day I stumbled upon the Timer bot on Twitter. I thought, with Grails I really can do that in 10 minutes! So here it is, and you can do it in 10 minutes as well!

The way a twitter bot works

If you are already a Twitter geek, you can well skip to the next part. But if not, let’s first understand how an interactive bot on Twitter like the timer bot works.

Timer is a bot that can remind you with your message in a pre-defined amount of time. e.g. You send a direct message over Twitter to Timer like “d timer 45 call mom”, and it will send you a direct message back 45 minutes later, telling you “call mom”.

There are actually two ways to interact with Twitter.

  1. Using the REST API
  2. Talking to the Twitter IM bot directly over one of the supported IM networks.

We are talking about “interactive” bots here, which means that you can send some commands to it, and it reacts to your command. So we need to use the second approach. And since one can connect to Google Talk using the open XMPP protocol, talking to Twitter over Google Talk is apparently the better choice.

And here are the steps for some paperwork.

  1. Register a new user on Twitter to act as the bot account. Let’s name it “mytimer”.
  2. Log on as “mytimer”, and in the settings, add the Google Talk account which you are going to use for the bot to connect to Google Talk and “chat” with the Twitter IM bot. You need to log on to GTalk using this account and send the supplied verification code to the Twitter IM bot and get your GTalk account confirmed.

OK. Now we have our GTalk account bound to the Twitter bot account. We are going to write the code to make the bot alive.

The plan

Here’s how we are gonna do it.

  1. Connect to Google Talk service using the Smack XMPP library
  2. Listen to direct messages coming from the Twitter GTalk account twitter@twitter.com.
  3. When a direct message comes in, parse it and send the message back in the pre-defined amount of time.

A Twitter direct message sent over Google Talk looks like this:

Direct from {Sender_User_Name}:
{Message Body}

Reply with ‘d {Sender_User_Name} hi.’

A Twitter user name can only consist of letters, numbers, and “_”. So we can safely tokenize the message using spaces and newlines as separators. Also, the first and last line of the message are of the fixed format.

Our timer command spec is, described in regular expression:

^\d+(s|m|h)*\s.*$

e.g. 5m buy the milk

This command means to remind in 5 minutes with the reminder “buy the milk”. Accordingly, appended after the first number, “s” means seconds and “h” means hours.

The code monkey job

Now let’s see the code. First define the service settings in Config.groovy.

//File: Config.groovy
chat {
    serviceName = "gmail.com"
    host = "talk.google.com"
    port = 5222
    username = "bot@your.domain" //This is the Google Talk account you prepare for the bot
    password = "your_password" //The Google Talk password
}

Then create a service to hold the code, say, TwitBotService. It looks like this.

//File: TwitBotService.groovy
import org.codehaus.groovy.grails.commons.ConfigurationHolder as C
import org.jivesoftware.smack.Chat
import org.jivesoftware.smack.ConnectionConfiguration
import org.jivesoftware.smack.Roster
import org.jivesoftware.smack.XMPPConnection
import org.jivesoftware.smack.packet.Message
import org.jivesoftware.smack.PacketListener
import org.jivesoftware.smack.filter.PacketFilter
import org.jivesoftware.smack.filter.PacketTypeFilter
import org.jivesoftware.smack.packet.Message
import org.jivesoftware.smack.packet.Packet
import org.jivesoftware.smack.PacketListenerclass

TwitBotService {
    boolean transactional = false
    XMPPConnection connection

    def connect() {
        ConnectionConfiguration cc = new ConnectionConfiguration(
        C.config.chat.host,
        C.config.chat.port,
        C.config.chat.serviceName)
        connection = new XMPPConnection(cc)
        PacketFilter msgFilter = new PacketTypeFilter(Message.class)

        def myListener = [processPacket:{ packet ->
            log.debug "Received message from ${packet.from}, subject: ${packet.subject}, body: ${packet.body}"
            def msg = packet.body
            if(!msg) return
            def words = msg.tokenize(' \n')
            //Direct message body
            def body = words[3..-6]
            //The Twitter user who sent the command, and we are going to reply to
            def to = words[2][0..-2]
            //The amount of time delay
            def delay = words[3]
            switch(delay[-1]){
                case 's':
 	       log.debug "Delay in seconds"
 	       delay = delay[0..-2].toInteger()
 	       break
               case 'm':
 	       log.debug "Delay in minutes"
 	       delay = delay[0..-2].toInteger()*60
 	       break
 	  case 'h':
 	       log.debug "Delay in hours"
 	       delay = delay[0..-2].toInteger()*3600
 	       break
 	  default:
 	       //default unit is minute
 	       delay = delay.toInteger()*60
            }
            //The reminder text to send back
            def reminder = body[1..-1].join(' ')
            if(delay instanceof Integer){
                new Timer().runAfter(delay*1000){
 	       //Send back direct message 'd user message'
 	       def cmd = 'd ' << to << ' ' << reminder
 	       sendChat(packet.from, cmd.toString())
 	   }
            }
       }] as PacketListener
       try {
            log.debug "Connecting to server..."
            connection.connect()
            connection.login(C.config.chat.username,C.config.chat.password)
            connection.addPacketListener(myListener, msgFilter)
            log.debug "Connected to server"
       } catch (Exception e){
        	log.error "Connection failed: $e.message"
       }
    }

    def disconnect(){
        connection.disconnect()
    }

    def sendChat(String to, String msg) {
        try{
            Chat chat = connection.chatManager.createChat(to, null)
            def msgObj = new Message(to, Message.Type.chat)
            msgObj.body = msg
            chat.sendMessage(msgObj)
        } catch (Exception e) {
            log.error "Failed to send message"
        }
    }
}

And finally, to start the bot with your grails app, simply add a call to TwitBotService#connect() in BootStrap.groovy.

//File:BootStrap.groovy
    def gTalkService
    def init = { servletContext ->
        gTalkService.connect()
    }
    def destroy = {
        gTalkService.disconnect()
    }

So here it is, your own timer bot recreated (with a bit more features) in 10 minutes, quick and dirty. And with the same approach, you can create all sorts of interactive Twitter bots.

P.S. Why would you create a Twitter bot while you can create a Google Talk bot in the same way?

Well, a GTalk bot can only interact with online GTalk users. But by leveraging the Twitter platform, users can interact with your bot either online with GTalk, or offline with mobile SMS. And that all comes with no cost at all. Ain’t that cool?

Deploying Grails - Things I learned from Feedlr

March 10th, 2008

When I was about to deploy Feedlr on my VPS server there wasn’t much information about the deployment of Grails out there. I know that deploying a Grails war is just like a Java war. But are there going to be any differences? How much memory do I need? Will it eat any more hardware than a plain old Java app? I really have no idea. So this post is about the server stuff I’ve learned by doing the deployment for Feedlr.

I started by deploying my Grails app on Tomcat 6 on a 256M RAM VPS. And since my blogs are also hosted on this VPS, I use mod_proxy_ajp to connect Tomcat to Apache for Feedlr.

My first attempt on the 256 VPS turned out very badly. When I started Tomcat with Feedlr deployed, the server simply becomes very unresponsive and swap activity started flying. That’s a bad sign for “low memory”, I think. So I upgraded the server to 512M RAM and things began to look normal.

Then comes the optimization part. I really don’t have much server tuning experience, especially with a limited resource VPS. So all this is new and interesting to me, solving issues one by one.

The first bad thing happened was that whenever Tomcat ran for a while, like a couple hours, the whole VPS box began to eat a low of swap, and connection will often timeout. I suspected Tomcat, but it turned out to be Apache. The default installation on Ubuntu for Apache + PHP5 uses the Apache mpm_prefork module. The problem is that mpm_prefork spawns multiple Apache instances to handle requests to the server. And the default config is like

<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 0
</IfModule>

The default config worked fine when I was only hosting a couple blogs on the 256 box. But with a Grails app on Tomcat running, memory becomes critical. So then I changed the settings to smaller numbers and it has been working fine now.

<IfModule mpm_prefork_module>
StartServers 2
MinSpareServers 2
MaxSpareServers 4
MaxClients 150
MaxRequestsPerChild 0
</IfModule>

I will replace mpm_prefork with mpm_worker module since the latter is more efficient for concurrency. Aptitude on Ubuntu can’t figure out for you how to put mpm_worker and PHP5 together so there are some config work to do. Currently I’m running mpm_worker on my development/staging server and it’s working fine but I just don’t know how it’s gonna behave on a low memory environment. BTW. If you have anything to say about this I’m all ears.

Then it’s about Tomcat. Actually Tomcat is working great with Grails out of the box. The tuning is all about memory and efficiency. Here’s my Tomcat startup opts:

CATALINA_OPTS=”-server -Xmx420m -Xms420m -XX:MaxPermSize=128m -Dcom.sun.management.jmxremote”

I set the heap size to 420M since I want to leave some room for my blogs. And so far this setting has worked fine for me. The “-Dcom.sun.management.jmxremote” part is for LambdaProbe which I’m using as the server management app.

Besides, for the production environment, it’s better to set reloadable=”false” for the Tomcat Context of the app. It should be false by default. In this way new code won’t be reloaded on the fly, but it saves quite some server overhead which can be notable on a low memory environment.

And finally, server logging. Here’s my logging settings in Config.groovy. Log files will be saved daily for stdout and stacktrace.

log4j {
    appender.stdout = "org.apache.log4j.ConsoleAppender"
    appender.'stdout.layout'="org.apache.log4j.PatternLayout"
    appender.'stdout.layout.ConversionPattern'='[%r] %c{2} %m%n'

//    	rolling file logger
    appender.logfile = "org.apache.log4j.DailyRollingFileAppender"
    appender.'logfile.File' = "/path/to/public_html/feedlr.com/logs/feedlr_grails.log"
    appender.'logfile.layout' = "org.apache.log4j.PatternLayout"
    appender.'logfile.layout.ConversionPattern' = '%d{[ dd.MM.yy HH:mm:ss.SSS]} [%t] %-5p %c %x - %m%n'

    appender.errors = "org.apache.log4j.DailyRollingFileAppender"
    appender.'errors.layout'="org.apache.log4j.PatternLayout"
    appender.'errors.layout.ConversionPattern'='%d{[ dd.MM.yy HH:mm:ss.SSS]} [%t] %-5p %c %x - %m%n'
    appender.'errors.File'="/path/to/public_html/feedlr.com/logs/feedlr_grails_stacktrace.log" //prod setting
    rootLogger="error,stdout"
    logger {
        grails="info,stdout,logfile" //prod settings
        StackTrace="error,errors"
        org {
            codehaus.groovy.grails.web.servlet="info,stdout,logfile"  //  controllers
            codehaus.groovy.grails.web.pages="info,stdout,logfile" //  GSP
            codehaus.groovy.grails.web.sitemesh="info,stdout,logfile" //  layouts
            codehaus.groovy.grails."web.mapping.filter"="info,stdout,logfile" // URL mapping
            codehaus.groovy.grails."web.mapping"="info,stdout,logfile" // URL mapping
            codehaus.groovy.grails.commons="info,stdout,logfile" // core / classloading
            codehaus.groovy.grails.plugins="info,stdout,logfile" // plugins
            codehaus.groovy.grails.orm.hibernate="info,stdout,logfile" // hibernate integration
            springframework="off,stdout,logfile"
            hibernate="off,stdout,logfile"
        }
    }
    additivity.StackTrace=false
}

P.S. I’m using SliceHost’s VPS and it’s great. I especially love the on-demand slice resizing which will automatically resize your VPS in minutes, with data completely intact. So you can totally start with a frugal setup and grow as needed. And did I mention completely full control over your server? I won’t say more but I know your inner geek would love it once you use it. So if you are seeking some good server space for your project why not give it a try with my referral link :)

Feedlr is now alive!

March 1st, 2008

Hi friend, my latest pet project feedlr has gone alive! After testing the initial version with friends, I’ve decided to make Feedlr open to the public now.

Feedlr is a service inspired by twitterfeed, built with Grails. It is both an automatic feed tweeting service like twitterfeed, and a bot showcase for miniblog users to browse, find, and follow interesting bots on popular miniblog services. Currently, Feedlr supports miniblogs including Twitter, JiWai, and FanFou. The idea is to make Feedlr a thriving farm of miniblog bots where anyone can create or adopt cute bots.

You can use Feedlr to broadcast feeds to multiple miniblogs

To do this, please first sign in or sign up for an account, and then click the “New” tab on the top and follow the simple wizard.

The wizard will ask for “bot accounts”, which means miniblog accounts used to broadcast the feeds. You should have already registered an account on a supported miniblog and provide the account information to the wizard.

A note on privacy here. Since the way miniblog API’s are designed, Feedlr has to have your bot login information in order to do the posting. So it’s strongly recommended that bot creators don’t use their personal accounts as bot accounts, but create dedicated accounts for the bot service only.

Feed creators can choose to attach a feed up to 2 bots at a time, choosing among the 3 currently supported services.

Feedlr is also a bot showcase

In addition, you can simply browse for bots on Feedlr and find interesting ones to follow. Once you want to follow a certain bot, there are two ways to follow it and both should take less than 10 seconds.

First, you can simply click the bot avatar and visit the bot’s homepage on the miniblog. Then just follow the bot as how you do it with the miniblog service.

Second, you can click the “Follow” hover menu. You will be prompted to enter your own login for the miniblog service where the bot is hosted. To ensure privacy and security, Feedlr does not record you login in any way. It’s strictly used to follow the bot for this one time.

Feedlr loves feeds, miniblogs, and you!

So start by feeding it with more feeds:)

If you have any comments, find any bugs, want any new features, or just want to say something, please leave a word or two by replying to this blog post. Feedlr is a new born baby. Love and caring makes it grow healthily and fast!

Streamlining JSON processing with Prototype 1.6

February 9th, 2008

The other day I was doing some refactoring. I became aware that with Prototype 1.6, there comes a nice Ajax.Response object as the first argument of all Ajax callbacks. And as the documentation points out, the “responseJSON” property is “The JSON body of the response if the content-type of the request is set to application/json. null otherwise.”

I used to process the JSON responses from Grails the old way using the “response.responseText.evalJSON()” thing. So it looks like this process is nicely streamlined now. And now that I’ve replaced the stock prototype in Grails with the new 1.6 version, this is a low hanging fruit to make the code nicer. So why not?

In your Grails controller code, simply replace all occurrences of

render(contentType:"text/json", ...

With

render(contentType:"application/json", ...

And in your javascript, replace all

response.responseText.evalJSON()

with

response.responseJSON

Congratulations! Your code has just become one step more concise and prettier.

So here’s just a little tip I feel like sharing with you.

P.S. Grails 1.0 has been out a couple days now. Great news for all!

Undocumented goodies of Grails - Part 1

January 29th, 2008

Working with Grails you can often discover exciting features from time to time. And even better (or worse?), some of them are undocumented, which feels kind of like treasure hunting.

Lately I discovered this gem of feature which is left out of the latest documentation. You can use a closure with the <g:select> tag to fully customize the displayed values of your options list!

It works like this. For example you have a list of values indicating minutes, e.g. 15, 30, 60, 120, … You want to give the list to <g:select>, but you don’t want to simply display the bunch of numbers as-is because that’s ugly. You want the minute values to be displayed as “15 Minutes”, “30 Minutes”, “1 Hour”, “2 Hours”, etc.

How would you do this? Custom tag lib? It sure works, but definitely will cost you more than a one-liner. But using a closure to supply the formatting logics directly to <g:select>, this IS a one-liner!

<g:select from='${minutesList}' optionValue='${{it<60?("$it Minutes"):(it>60?("${it/60} Hours"):("${it/60} Hour"))}}' name='prettyList'/>

That’s it! The closure supplied to optionValue takes the List given by the “from” attribute, and iterates the List. Now your controller can take the raw minutes values while the front end displays human friendly minutes and hours reading.

By reading the <g:select> source code, you will see that the attribute “optionKey” can take a closure, too. What clever use of this can you think of now?

Update: I think part of the reason this is undocumented could be that this would encourage bad habits of littering the views with too much logic. So use it wisely. I recommend only using this for simple and clear one-liners.