Comments for Sujee Maniyam's Blog

Comment on [tech] Hbase Map Reduce Example by Sujee Maniyam

Sujee Maniyam — Thu, 17 May 2012 18:17:31 +0000

Hbase APi changed a bit recently. Thanks for pointing this out. I will update the code.
regards
Sujee Maniyam (admin)

Comment on [tech] Hbase Map Reduce Example by k A r T h I k

k A r T h I k — Thu, 17 May 2012 17:12:42 +0000

Perfect!
It works. I’ve used your program in the following enviroment:
Hadoop 1.0.2
HBase 0.92.1

I observed a few changes:
–HBaseConfiguration hbaseConfig = new HBaseConfiguration();
++Configuration conf = HBaseConfiguration.create();

–String columns = “details”; // comma seperated
–scan.addColumns(columns);
++scan.addFamily(Bytes.toBytes(“details”));

Need to replace — with ++ statements in your program.
Thanks a lot.

Comment on Performance testing a HBase cluster by minsbrz

minsbrz — Mon, 14 May 2012 05:43:08 +0000

I have been testing on HBase for performance like you.
But the result was not same as you , How can i do that .
My testing result is about 4000 TPS on 4 clusters ,
memory write only is 10000 TPS,
could you tell me detail the property of xml and status of writing ,to memory or to Disk(flush,compact act so on..) ?
Thanks in advance for advice.

Comment on Performance testing a HBase cluster by minsbrz

minsbrz — Wed, 09 May 2012 00:54:12 +0000

Hi.It is very great performance to me.
Can I see the configuration of hadoop & hbase.
We have been testing that tools , but the result of testing is not same.
so we want to see detail option .
please give me advice. thanks for reading.

Comment on [tech] Ruby on Rails CRONjobs by Luis Castillo

Luis Castillo — Mon, 30 Apr 2012 21:16:58 +0000

Thansk, this tutorial help me a lot!!!

Comment on [tech] Hbase Map Reduce Example by Ramanjaneylu

Ramanjaneylu — Sun, 29 Apr 2012 09:08:05 +0000

Hi sujee,
Wonderful doc which helped me a lot.

I want some suggestions like,
1) how to display output of reduce functions into JTable.
2) I have a jar file, after execute that jar file a text box will display. how the data which we entered in text box will get output for that data.

Please guide me, Its urgent

Thanks & Regards,
V.Ramanjaneylu.

Comment on About/Contact by Bhavesh Shah

Bhavesh Shah — Wed, 25 Apr 2012 06:56:38 +0000

Hello,
I have a task of data processing. I want to import the large data from SQL Server into Hive using SQOOP. I want to process the data and generate some result. For that, I have implemented some task for it in Hive using Hive JDBC.

Now I want to deploy this task on Amazon EMR. I have read some links related to AWS EMR for launching the instance and about what is EMR, How it works and etc…

I have some doubts about EMR like:
1) EMR uses S3 Buckets, which holds Input and Output data Hadoop Processing (in the form of Objects). If I use SQOOP to import the tables from SQL Server then it will stored in the form of File.
2) As already said I have implemented a task for my use case in Java. So If I create the JAR of my program and create the Job Flow with Custom JAR. Will it be possible to implement like this or do need to do some thing extra for that?
3) As I said in my Use Case that I want to export my result back to Ec2 with the help of SQOOP. Does EMR have support of SQOOP?
4) And if supported then I have one doubt as:
I am using Hive. Whatever Tables I created or Imported using SQOOP that will
automatically stored in HDFS (in /user/hive/warehouse directory). Then If I use
S3 to stored the imported data from SQL Server then How can I make link with
HDFS (/user/hive/warehouse directory) so that Hive can show/access the tables
And also I want to import the tables from SQL Server daily/weekly so I will use
S3.

How to overcome from all this stuff? I am new to Amazon Services and I am not getting how to proceed for this? Please help me.

Comment on [tech] Hbase Map Reduce Example by ramya

ramya — Tue, 17 Apr 2012 10:03:16 +0000

Hey
Can you help me out a lil more on the map function. I understood the basics of mapreduce but can’t get how to utilize it when you are doing more complex processing.
How do we actually access the individual columns in a map function? I know that a map returns a row. how do we break it down so that we can compare values of two or more columns which maybe of different column families?

For example, a User has a visitor and I want to know if that visitor is one of his friends?

Please help.Just started learning hbase.Having a problem is visualizing how the data appears in a map reduce.

Comment on [tech] Hbase Map Reduce Example by Arati Patro

Arati Patro — Thu, 12 Apr 2012 12:07:57 +0000

Hi,

Can you please how explain you created a composite row key.
I have been trying to create an HBase Table with a composite row key that has a number and a timestamp. The Put Constructor you have used takes in two byte values thereby converting the timestamp to a byte value as well. While I am able to do this, I’m unable to perform any operation on the timestamp or the number individually as the entire row key is inserted as one entity, Could you please guide further on how a partial key scan may be performed.
Thanks in advance.

Comment on [tech] sample .gitignore files for various projects by Nikola Kolev

Nikola Kolev — Sat, 31 Mar 2012 06:01:22 +0000

Great list indeed