Opengrok: Browser based search for your workspace

Installing OpenGrok

Installing tomcat

Scripts to monitor the changes

Installing Sun JDK 7 on Ubuntu

Given that Oracle does not have a release for Ubuntu platform, one way to seamlessly install the latest version is

$ sudo add-apt-repository ppa:webupd8team/java

$ sudo apt-get update

$sudo apt-get install oracle-java7-installer

Added after inputs from Kiran for Centos

wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F" "http://download.oracle.com/otn-pub/java/jdk/7u51-b13/jre-7u51-linux-x64.rpm"

A curious case of Hbase

Screen way of terminal management

My frequently used command to connect to screen? 

screen -dxR work

What is screen?

It is  a terminal multiplexer, essentially giving the ability . man screen is the best option still.

Getting started video can be useful.

Why screen?

Its original reason for existence was allowing you to switch between sub-sessions on a video-display terminal, but it grew to allow sessions that could be detached and reattached (if you went home for the day, or say you were connecting via a glitchy network) and eventually to allow the same session to be simultaneously accessed from multiple places.

Real power of screen

1. Persistence of the terminal based work. 
2. Start your builds -- go home and see all the results intact. Resistant to laptop or network snags.
3. Unlimited windows (unlike the hardcoded number of Linux virtual consoles)
4. Scroll-back buffer (not limited to video memory like Linux virtual consoles) & Logging.
5. Copy/paste between windows.
6. Notification of either activity or inactivity in a window
7. Split terminal (horizontally and vertically) into multiple regions
8. Locking other users out of terminal.

Advanced users of screen

Buffering

Ctrl +a+ [  --------- Page or arrow keys work best

Ctrl +a+ ]  --------- To close the buffer read session

Helpful on remote machines where the logs are being tail-ed.

Terminal Scrolling

Though the above buffering solution of (Ctrl+a+[ ) helps, screen uses alternate screen. So one way is to tell screen not to use this. Scroll-back can also be achieved on the terminal using
‘termcapinfo xterm ti@:te@’ to your .screenrc file.

Starting and Naming sessions

screen -S  

Reconnecting to screen session

screen -x

Split Screen

You can split your terminal windows horizontally and vertically (for vertical split, you need to patch your screen).

To Split the screen horizontally: Ctrl+a & S

To split the screen vertically: Ctrl+a & | or Ctrl+a & V ( the screen has to be patched for this)

To switch between windows: Ctrl+a & Tab

To kill your current window: Ctrl+a & X

Logging

The best thing what I like about screen is that you can have logs of anything you have done on screen, which can be used in lot of things in future. To enable logging, use “Ctrl+a” followed by “H“. This will create a file with name “screenlog.0” in your home directory, which will contain anything or everything you have done on the screen. If you want to stop logging, use the same, “Ctrl+a” & “H“.

You can also enable logging while starting your screen session, which can be done by using “-L” switch:

screen -L -S

Copying and Pasting across terminals 

To enter copy mode in screen, hit Ctrl+A, then [. You can now use the arrow keys, or vim-like keybindings, to move around the screen. 0 gets you to the start of a line, and $ to the end of a line.

• When you've reached the point where you want to start the copy, hit Enter.
• Move the cursor to the end point of your selection (you'll see what you're copying highlighted), and then hit Enter again.
• Now move the cursor to wherever you want to paste the selection (you can move to another screen within your session), and hit Ctrl+A ] to paste the selection.
• You can also use Ctrl+A > filename to paste the selection to a filename and Ctrl+A < filename to read a file into the selection buffer so you can then paste it out again using Ctrl+A ].

When in copy mode, you can also use screen's scrollback feature (i.e., you can scroll back upward past the top of the currently displayed text) and the search function.

Command line Ctrl+a does not work in screen?

It works in a different way where ctrl+a+a will get the effect.

Searching in the buffer (Like VIM )

When in Buffer mode (Ctrl + [) use / for backword search and ? for forward search. 

Getting ..... lines on the screen terminal

You see this when the user is connected in a multi-user mode and 2 users are on the same terminal. To overcome this, first issue screen -d and then connect using screen -x  

Increasing the scroll back limit

defscrollback 1000

This increases the scrollback to 1,000 lines; you can edit this value as you prefer.

Password-less ssh login

ssh from machine1 to machine2 should be password-less (say for some automation tasks)

user1@machine1>pwd
/home/user1/.ssh/
user1@machine1> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user1/.ssh/id_rsa):
Created directory '/home/user1/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user1/.ssh/id_rsa.
Your public key has been saved in /home/user1/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 user1@machine1
user1@machine1>cat ~/.ssh/id_rsa.pub | ssh user1@machine2 'cat >> ~/.ssh/authorized_keys && chmod 640 ~/.ssh/authorized_keys'
user1@machine2's password:
user1@machine1>ssh user1@machine2
user1@machine2>hostname
machine2

Mail thru javascript

you can automate sending bulk emails. In case you dont want to send BCCs or even CCs, (some rare cases, i agree)..

if your content is all text or html, (the example given is for text), and the mail-ids you need to send individual mails are one per line in a google spreadsheet (customisable to add other data),
you can open the spreadsheet, go to tools->ScriptEditor
and start typing your javascript! heres an example that sends each mail to one individual at a time, in clear text

//This thing just adds a convenient menu/submenu in your spreadsheet window!


function onOpen() {
  var subMenus = [];
  subMenus.push({name: "Email File to selected Range", functionName: "mysendmail"});
  SpreadsheetApp.getActiveSpreadsheet().addMenu("Email", subMenus);
}

function mysendmail() {
  var numAddrs=SpreadsheetApp.getActiveRange().getValues().length;

  for (var i=0; i
    var toAddr=SpreadsheetApp.getActiveRange().getValues()[i].toString();
    //Browser.msgBox("what is in toAddr is: " + SpreadsheetApp.getActiveRange().getValues()[i].toString());

// this is the line that sends the mail
    GmailApp.sendEmail(toAddr, "sendmail", "testing");
  }
}

if you want to attach a file, you need to use MailApp's sendEmail interface that accepts the attachment.

MailApp.sendEmail(recipients, emailSubject, emailMessage, {attachments: fileName});

check the documentation for other cool scripting interfaces with google apps.

RSS feed for gmail ...

I like this feature, where gmail is now accessible over rss feed.

try this in firefox ( is rss feed support there in chrome by now? )

https://USERNAME:PASSWORD@gmail.google.com/gmail/feed/atom

Though this feature is nice , i wonder if there are any uses of this feature in-spite of PoP3 or SMTP clients

linux system info

if you ever want to get low level h/w info from your linux installation and your proc file system is cryptic, use 'biosdecode' and/or 'dmidecode'

biosdecode is known to be inadequate. some recent versions of enterprise linux dont ship biosdecode by default. use dmidecode instead. however, it is known to be inaccurate or un-reliable. But, it gives a quick and easy peek into the system and if you need reliable data, you must anyway look into the proc fs.

'dmidecode -t' prints all available sub-system level info.

New features in Java 7: Switch on Strings

At last there is this feature is in Java 7

String s = "something something something"
switch(s) {
case "Jayanagar":
   goToOxford(s);
   break;

 case "Rajajinagar":
 case "Navrang":
   haveDosainAjanta(s);
   break;

 default:
   goToBrahminsCoffeebar(s);
   break;
}
More on ... http://java.sun.com/features/jdk/7/

Getting customized mobile updates from twitter and other RSS services

This post marks the 4th year of this blog.
Hope is to actively engage in writing this blog.

Presented below is an orchestrate of seemingly disparate technologies to enable your favorite updates on your cellphone.
Note: Your cellphone need not have Wifi or GPRS options. This works on basic SMS incoming facility.

Now some of my favorite online items which i need to keep track are:
1) Twitter updates from selected friends.
2) Posts from selected blogs which i want to keep myself updated.
3) Instant updates from Google alerts.

Here is one way we can have all the elements integrated:

1) Collect RSS feeds of all the items listed above.
2) Using yahoo pipes you can do the following:
2a) Using Fetch Feed , aggregate all the feeds.
2b) Use a sort by published date to get the feeds in order. Click here for a preview
2c) This service gives out a common aggregated RSS feed for the selected set.
2d) Also one can use alternatives like RSSmix to achieve a common RSS feed.
3) To get the updates on the cellphone, use Google SMS Channel . By configuring our cellphone for the alerts in Google SMS Channel , we can get the desired updates on our cellphone.

Java URL Bloomer

I came across an interesting problem in java



public class URLBloomer {
public static void main(String[] args) throws MalformedURLException, URISyntaxException {
String[] urls= {
"http://meshlabsinc.com",
"http://meshserver.org"
};
Set testing=new HashSet();
for (String url : urls) {
testing.add(new URL(url));
}
System.out.println(testing.size());
}
}


Output : 1

Reason .... meshlabsinc.com and meshserver.org point to the same IP


Lesson learnt is to use URI instead of URL class.


public class URLBloomer {
public static void main(String[] args) throws MalformedURLException, URISyntaxException {
String[] urls= {
"http://meshlabsinc.com",
"http://meshserver.org"
};
Set testing=new HashSet();
for (String url : urls) {
testing.add(new URI(url));
}
System.out.println(testing.size());
}
}

Output : 2

Getting the create table command for an already existing table in mysql

mysql>SHOW CREATE TABLE

mysql> create table x (i integer);
Query OK, 0 rows affected (0.27 sec)

mysql> show create table x;
+-------+--------------------------------------------------------------------------------------+
| Table | Create Table |
+-------+--------------------------------------------------------------------------------------+
| x | CREATE TABLE `x` (
`i` int(11) default NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1 |
+-------+--------------------------------------------------------------------------------------+
1 row in set (0.06 sec)

mysql>

Java code to send message to your gtalk friends

Have used a library called Smack (http://www.igniterealtime.org/projects/smack/index.jsp)

import java.util.Collection;

import org.jivesoftware.smack.*;

public class GtalkClient {

public static void main(String[] args) throws XMPPException {
ConnectionConfiguration config = new ConnectionConfiguration("talk.google.com", 5222, "gmail.com");
XMPPConnection connection = new XMPPConnection(config);
connection.connect();
connection.login("< username >","< password >");

// Below is the code to get the users


// Roster roster = connection.getRoster();
// Collection rosterEntries = roster.getEntries();
//
// System.out.println("\n\n" + rosterEntries.size() + " friend(s):");
// for(RosterEntry rosterEntry:rosterEntries)
// {
// System.out.println(rosterEntry.getUser());
// }

// Here is a code to send message to a friend

MessageListener messageListener = null;
Chat chat = connection.getChatManager().createChat("shantanu.gg@gmail.com",messageListener);
chat.sendMessage("Hello this is a ping from a java program");
}
}

Sending email using GMAIL using Java Mail Api

Note: Search for changeme and replace with appropriate terms.





import javax.mail.*;
import javax.mail.internet.*;
import java.util.*;

public class MailSender {
final String senderEmailID = "changeme @ changeme .com";
final String senderPassword = "changeme";
final String emailSMTPserver = "smtp.gmail.com";
final String emailServerPort = "465";
String receiverEmailID = null;
String emailSubject = null;
String emailBody = null;

public MailSender(String receiverEmailID, String emailSubject, String emailBody) {
this.receiverEmailID=receiverEmailID;
this.emailSubject=emailSubject;
this.emailBody=emailBody;


Properties props = new Properties();
props.put("mail.smtp.user",senderEmailID);
props.put("mail.smtp.host", emailSMTPserver);
props.put("mail.smtp.port", emailServerPort);
props.put("mail.smtp.starttls.enable", "true");
props.put("mail.smtp.auth", "true");
// props.put("mail.smtp.debug", "true");
props.put("mail.smtp.socketFactory.port", emailServerPort);
props.put("mail.smtp.socketFactory.class",
"javax.net.ssl.SSLSocketFactory");
props.put("mail.smtp.socketFactory.fallback", "false");

SecurityManager security = System.getSecurityManager();

try {
Authenticator auth = new SMTPAuthenticator();
Session session = Session.getInstance(props, auth);
// session.setDebug(true);

MimeMessage msg = new MimeMessage(session);
msg.setText(emailBody);
msg.setSubject(emailSubject);
msg.setFrom(new InternetAddress(senderEmailID));
msg.addRecipient(Message.RecipientType.TO,
new InternetAddress(receiverEmailID));
Transport.send(msg);
} catch (Exception mex) {
mex.printStackTrace();
}


}
private class SMTPAuthenticator extends javax.mail.Authenticator {
public PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(senderEmailID, senderPassword);
}
}
/**
* @param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
MailSender mailSender=new MailSender("changeme @ changeme .com","Test Mail from Puretechie","Here goes the body");
}

}

Common bad practice in database calculations

create table x ( i int, j varchar(10))
insert into x values (1,'a')
insert into `x`(`i`,`j`) values ( '2',NULL)


select count(*) from x
2
Select count(j) from x
1

Note the difference when the count(col_name) is used.

count(col_name) is used under the impression that it is faster than count(*) , however quite the opposite is true.

MYISAM table MySQL has cached number of rows in the table. Thats the reason why MYISM is able to instantly answer COUNT(*) query, but not COUNT(col_name).

Why ? Because say if col_name column is not defined as NOT NULL there can be some NULL values in it and so MySQL have to perform table scan to find out. This is also why result is different for the second query.

Using count(*) instead of count(col_name) falls in the best practice category.

Ways to manage RSS feeds

RSS feeds aim to manage the information load.

I for example blog at these places :

a) http://dinchari.blogspot.com/ --- my Personal blog
b) http://puretechie.blogspot.com/ --- were a group of like minded tech enthusiasts post about topics of their interest.

Ideally I would like my friends to have be updated with my thoughts on either blogs. Currently they have to refer to the individual rss feeds separately.


To ease matters, I have used Yahoo Pipes to create a combined RSS feed for the consumers as a single combined RSS feed.

Click here for the new combined RSS link


Now this essentially is simple.

We can develop on this use case extensively.

Initially,
a) Now in the tech blog: there are many other authors. With the above URL , the consumers will also get the updates from these other authors. There needs to be filtering mechanism based on Author.

Adding more complexity,
b) Suppose you like all my articles about the puzzles and are not interested in any other topics. This is where the whole thing gets tricky. Using filtering its possible, but I have to explore the capabilities of customizing to individual consumers.

Processing RSS feeds --the Python way

[shantanu@myjunkyard rss]$ cat a1.py
#!/usr/bin/env python
import feedparser
import sys
d = feedparser.parse(sys.argv[1])
for i in d['entries']:
for j in i['links']:
print i['title'] + "######" + j['href']


[shantanu@myjunkyard rss]$ cat rssFeed.list
http://puretechie.blogspot.com/atom.xml

[shantanu@myjunkyard rss]$ for i in `cat rssFeed.list `; do ./a1.py $i; done

See it Yourself :)

Ruby on Rails --- it time

After many futile attempts RoR finally is happening this year. Though I tried my best to use Java for the project(as usual , cox thats a comfort zone) , somehow winds changed direction.

For statistics The TIOBE Programming Community index which gives an indication of the popularity of programming languages, ranks Ruby 10th in popularity with 2.66% of programmers.


My initial thoughts

1) There is a need to think in a language, understand its philosophy. An interesting story to share from RoR for dummies book about the name Ruby on Rails. Since the year 2000, teams of Java programmers have been using a framework named Struts. But the word strut means something in the construction industry. (A strut is a horizontal brace, and a sturdy one at that.) Well, a rail is also a kind of horizontal brace.
And like Ruby, the word Rail begins with the letter R. Thus the name Ruby on Rails.

2) Understanding MVC is ingrained
http://wiki.rubyonrails.org/rails/pages/UnderstandingRailsMVC

Best part of the language is that it forces MVC , similar to Java forcing OO.

These are still baby steps , but has been exciting.

Powerset is released today

Powerset is a first of its kind .... a context sensitive natural language based search engine.
Currently powered by Wikipedia, this is a potentially disruptive technology.


Simple queries like


Where is PESIT?
or
vice chancellor of Visvesvaraya Technological University


Interesting results!!!!

Punishment for 3 days.

For 3 days I was not able to access my system with my ID.

I kinda circumvented the problem by allowing remote root ssh to my machine(which is criminal).

The problem was this:
root@shantanu>ssh sg202516@shantanu
Password:
Last login: Fri Apr 11 17:19:51 2008 from shantanu
NO LOGINS: System going down in 30 seconds
Connection to shantanu closed.
root@shantanu>

Untill today I was ok with it, but I could not access my screen sessions remotely.

So finally decided to debug it.

The issue was the presence of a file called /etc/nologin

root@shantanu>cat /etc/nologin
NO LOGINS: System going down in 30 seconds

Removing the file solves the crap.

With a feeling of achievement I told my neighbor about this.

The reply came back "Check the date of the creation of the file. It should be Apr 8th. You had not locked the system!!!"

Related Old link:http://dinchari.blogspot.com/2006/07/assholes-learn-this-way_27.html