Tuesday, November 26, 2013

Using Postgresql in a Grails Application

I've always been using MySQL with almost all my web applications, but I've read a few articles talking about Postgresql and decided to give it a try. Being so used to MySQL, I wanted to see how fast it would take to at least be able have usable knowledge of Postgresql. So, let get to it!
  • Install Postgresql on Ubuntu 13.10
  • Create Users and database on Postgresql
  • Configure a grails application to use Postgresql
  • Verify application is connecting to the Database

Install Postgresql on Ubuntu 13.10

There's a number of tutorials on how to install postgres. Some included adding a new repository, but I tend to prefer the packages that is associated with an Ubuntu version. In my case since I am using Ubuntu 13.10, the version available is postgresql 9.1

  sudo apt-get install postgresql-9.1  
If you have a different version of Ubuntu, you can search for the postgresql version using the command:
  apt-cache search postgresql  

Create Users and database on Postgresql

Login to Postgresql. Similar to MySQL, you would want to create a user for your Grails application that would have access to the Postgresql database. In order to do this, you need to login as the default Postgresql user, which is 'postgres'

 sudo -u postgres psql template1  
Change the password for the default user and exit from the psql command-line. Note that '\q' is the command to quit the psql command-line.
 template1=# ALTER USER postgres with encrypted password 'newpass';  
 template1=# \q  

Modify a configuration file called pg_hba.conf and change the line below - changing 'peer' to 'md5'.
 unix$ sudo vi /etc/postgresql/9.1/main/pg_hba.conf  
   local all postgres peer md5

Restart Postgresql.
 sudo /etc/init.d/postgresql restart  

Create the new user that would be used in your Grails application. To know what options are available for createuser.
 unix$ createuser -U postgres -d -e -E -I -P -r -s myuser<br/>  
 Enter password for new role: <br/>  
 Enter it again: <br/>  
 Password: <br/>  
 CREATE ROLE myuser ENCRYPTED PASSWORD 'md5eb6108579123bee126149054337490' SUPERUSER CREATEDB CREATEROLE NOINHERIT LOGIN;  

Restart the database again using the command above ('postgresql restart');

Verify if the new user was created, login using the username and password. You should be able to login with your new user and password and see the command-line where you can issue SQL queries.
 unix$ psql -U myuser -d mydb -W  
 Password for user myuser:  
 psql (9.1.10)  
 Type "help" for help.  
 mydb=#   
Create a new database that you would use for your Grails application.
 unix$ createdb -U myuser -W mydb  

Configure a grails application to use Postgresql

After you have successfully installed and configured Postgresql, in your Grails application, you need to modify 2 configuration files, BuildConfig.groovy and DataSource.groovy.

First, we need to add the JDBC jar for Postgresql. Add this line inside the BuildConfig.groovy file inside the dependencies. Verify the jar version that is available on maven.
 dependencies {  
     // specify dependencies here under either 'build', 'compile', 'runtime', 'test' or 'provided' scopes e.g.  
     //runtime 'mysql:mysql-connector-java:5.1.24'  
           runtime 'postgresql:postgresql:9.1-902.jdbc4'   
   }  

Next, add the following configuration in DataSource.groovy.
    dataSource {  
       driverClassName = "org.postgresql.Driver"  
      dialect = org.hibernate.dialect.PostgreSQLDialect  
      username = "myuser"  
      password = "mypassword"  
      dbCreate = "update" // one of 'create', 'create-drop','update'  
      url = "jdbc:postgresql://localhost:5432/mydb"  
     }  

Verify application is connecting to the Database

Run the Grails application (grails run-app) and verify that the tables are created. You can login again using psql and issue the command \dt to list all tables. You should be able to see all your Grails domain classes.
 unix$ psql -U myuser -d mydb -W  
 Password for user myuser:   
 psql (9.1.10)  
 Type "help" for help.  
 mydb=# \dt  
         List of relations  
  Schema |    Name    | Type |  Owner    
 --------+-------------------+-------+------------  
  public | table_one    | table | myuser  
  public | table_two    | table | myuser  
  public | table_three   | table | myuser  

So far so good. One problem I encountered is that my Grails app had a table called 'User'. Unfortunately, this is a reserved word in Postgresql. It was an easy fix by just mapping the domain class user to the table using 'static mapping'.

Tuesday, March 12, 2013

Grails Plugin for Yahoo Finance

I've recently experimented on taking web-services JSON response and automatically putting these into java object/s. So playing around with Yahoo YQL resulted in this Grails open-source plugin. It has been approved recently and is located in github.

Github - ibaralf YahooFinance

After installing the plugin, it gives the user several methods to retrieve company stock statistics as well as stock historical data.

In a service or controller:

You can call the methods available in the YahooFinanceYQLService.
  • yahooFinanceYQLService.quote(String symbols)
  • yahooFinanceYQLService.quote(String symbols, String statistics)
  • yahooFinanceYQLService.getHistoricalQuotes(String symbol, int numberOfDays)
  • yahooFinanceYQLService.getHistoricalQuotes(String symbol, String startDate, String endDate)
  • yahooFinanceYQLService.getHistoricalQuotes(String symbol, Date startDate, Date endDate)
  • yahooFinanceYQLService.availableStats - list of all statistics that can be retrieved.

NOTE: One thing peculiar with Yahoo web-services (especially their finance service), there seems to be 2 different web-services running, first - the Yahoo Query Language YQL and the other web-service that returns CSVs as explained here in gummy-stuff website. The plugin implements both, but ended up using the CSV service since it seems to be more reliable.

* IMPORTANT: PLEASE READ

* I do not guarantee the reliability or availability of the Yahoo web services.
* I am not qualified, licensed, approved or authorized to make any investment recommendations or give any such advice.
* All data and information on this site is based on data from sources that are not under my control.
* I cannot guarantee the accuracy of any of the data, in fact I know of some significant data problems such as, but not limited to, incorrect stock splits that lead to radically incorrect prices.
* No investing decision should be taken based on the information on this site alone, and none of the information here is a recommendation to buy or sell any security!!
* Do the appropriate further analysis and due diligence before investing.
* I make no guarantees whatsoever about future returns using the methods.

Tuesday, January 15, 2013

Getting Stock Historical Data using YQL

select * from yahoo.finance.historicaldata where symbol="GOOG" and startDate="2010-02-08" and endDate="2010-03-10"
http://developer.yahoo.com/yql/console/

Monday, October 22, 2012

Removing trailing dos carriage returns ^M

Sometimes when you're comparing text files, the lines looks identical but the diff command shows that the two lines are different. One reason might be because one file has dos carriage returns (^M) and the other doesn't. One easy way to check this is by opening the file using VI, ^M will show up at the end of the lines as shown:
There are several ways of removing this dos carriage return, but my preferred tool is sed. I had a problem getting sed to substitute this because you need to actually type CTRL-V CTRL-M when writing the regular expression.
    unix$ sed 's/^M//g' -i myfile.txt
NOTE: type CTRL-V CTRL-M for the pattern inside '/s/^M//g'. The above sed pattern deletes all occurrence of ^M and saves it on the same file using the -i argument.

Tuesday, October 16, 2012

MYSQL: How to get the summary of tables in a DB

I sometimes want to see the number of rows in the tables of my DB. Instead of using COUNT for each table, use this command to get the summary of your database.
   mysql> show table status; 
Results: +----------------------------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+ | Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment | +----------------------------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+ | account_customer | InnoDB | 10 | Compact | 0 | 0 | 16384 | 0 | 0 | 51380224 | 1 | 2012-10-16 12:34:33 | NULL | NULL | latin1_swedish_ci | NULL |

Thursday, September 13, 2012

configure: error: Your intltool is too old.

While installing a new application on Ubuntu 12.04, I got this error when I ran ./configure:

configure: error: Your intltool is too old. You need intltool 0.40.0 or later.

Seems like the error message is a bit inaccurate, you're probably missing this software package. You need to install it using apt-get.

ubuntu12.04$ sudo apt-get install intltool

This should fix the error.

BTW: I encountered this issue when installing a very nice small application called alarm-clock. What I like about it is that it can be set two different ways - a) have an alarm go off at a certain time, or b) have a timer count down. See snapshot below.

Friday, August 31, 2012

Useful links

This might be helpful to those that have cut their cable TV.

http://jarkhd.blogspot.com/2011/10/hacking-digital-cable.html