Writeup I sent to semih, Stanford University for my Giraph Tester project.

Question: How to write and run your own apache giraph code (Computation / InputFormat / OutputFormat)?

Answer: There are two ways according to my knowledge.

Way 1:

1. Write your custom computation code.

2. Copy that file in apache giraph source code's “giraph-examples” folder.

3. Compile whole giraph source code again with maven.

4. Use giraph jar to run your code, similar to example given on giraph quick start page.

Problem:

Very slow process.

Hadoop psudo-mode setup require.

Input and output files are in HDFS.

Everytime you need to compile giraph-source code with maven.

Way 2:

1. Use Eclipse or other IDE

2. Add jar files from Apache-Hadoop's lib folder & Apache-Giraph's lib folder in Build-path.

3. Write your custom inputFormat/outputFormat/Computation java code

4. Write your giraph Runner java code. (Giraph Job Runner file)

5. Run and test your code on single Click

Advantages:

Faster than Way1.

No Hadoop setup require.

Input and Output files are on local Systems only.

In development phase, we make lot of changes in our code and we need fast result on our sample test input file. So way-2 works better in this case.

Question: How we can debug in those two cases?

Answer: We have already discussed two Ways to write/run giraph code.

For way-1, semih and his team at Stanford University have developed tool called Graft and now it is part of Apache Giraph project.

For way-2, I am not sure if Graft can also work in this case. If not than we can build another debugger to work for way-2.

Question: How to approach for building debugger for Way-2?

Answer: Basic idea is to trace the state of vertice,edges & messages of each superstep, store them in JSON format and plot them using your custom Graph Visulization program.

Question: How much progress I have done for building debugger for way-2?

Answer: I have partially developed Graph Visulization program which is inspired by Graft. I have defined my own JSON format and using it to plot graphs according to coresponding supersteps.

Question: Where Am I stuck?

Answer: I am not able figure out on how to trace program. I must store trace in JSON format and visualize it.

I can either write my trace method which need to be called by user and use will feed current vertex & message status as parameter in it.

I may also change the original java source code of standard files(like BasicComputation) and repackage them in jar. So user must use my modified jar files instead of original giraph-lib jars.

JAVA Chat Application with Multithreading

Server Part:

import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Scanner;

/**
 *
 * @author nishant
 */
public class Server {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        // TODO code application logic here
     
     
        try
{
final int PORT = 6677;
ServerSocket server = new ServerSocket(PORT);
System.out.println("Waiting for clients...");

while (true)
{
Socket s = server.accept();

System.out.println("Client connected from " + s.getLocalAddress().getHostName());

Client chat1 = new Client(s,1);
                                Client chat2 =new Client(s,2);
Thread t1 = new Thread(chat1);
                                Thread t2 =new Thread(chat2);
t1.start();
                                t2.start();
}
}
catch (Exception e)
{
System.out.println("An error occured.");
e.printStackTrace();
}
    }
}
 class Client implements Runnable{

private Socket socket;
int id;
public Client(Socket s,int pid)

        {
            id=pid;
socket = s;
}

@Override
public void run()
{
try
{
Scanner chat = new Scanner(System.in);
Scanner in = new Scanner(socket.getInputStream());
PrintWriter out = new PrintWriter(socket.getOutputStream());

while (true)
{
                            if(id==1)
                            {
String input = chat.nextLine();
out.println(input);
out.flush();
                            }
                            if(id==2)
                            {
if(in.hasNext())
System.out.println(in.nextLine());
                            }
}
}
catch (Exception e)
{
e.printStackTrace();
}
}

}

Client Part:

import java.io.IOException;
import java.io.PrintWriter;
import java.net.Socket;
import java.util.Scanner;

/**
 *
 * @author nishant
 */
public class Client {
    private final static int PORT = 6677;
private final static String HOST = "localhost";

public static void main(String[] args) throws IOException
{
try
{

Socket s = new Socket(HOST, PORT);

System.out.println("You connected to " + HOST);

Clients client = new Clients(s,1);
Clients clients =new Clients(s,2);
Thread t1 = new Thread(client);
                        Thread t2 =new Thread(clients);
t1.start();
t2.start();
}
catch (Exception noServer)
{
System.out.println("The server might not be up at this time.");
System.out.println("Please try again later.");
}
}
}
 class Clients implements Runnable {

private Socket socket;
int id;
public Clients(Socket s,int pid)
{
socket = s;
                id=pid;
}


public void run()
{
try
{
Scanner chat = new Scanner(System.in);
Scanner in = new Scanner(socket.getInputStream());
PrintWriter out = new PrintWriter(socket.getOutputStream());

while (true)
{
                            if(id==1)
                            {
String input = chat.nextLine();
out.println(input);
out.flush();
                            }
                            if(id==2)
                            {
if(in.hasNext())
System.out.println(in.nextLine());
                            }
}
}
catch (Exception e)
{
e.printStackTrace();
}
}

}

JOMP: Sample Program, Setting ClassPath

Read More about JOMP from website of EPCC

Step1: Download "jomp1.0b.jar" File from Here

Step2: Put it into your directory for example "/home/nishant/JOMP"

Step3: To install JOMP , we just need to add it into CLASSPATH

           For this,
         
           goto your home folder.
           For example user is nishant. so go to nishant's home folder
           type "cd" only to go into user's home folder.
           now we need to edit ".bashrc" file of user.
           Here "." before file indicate that file is hidden (Linux Concepts :) )
         
           now add classpath in .bashrc file by adding following line,

           export CLASSPATH=$CLASSPATH:/home/nishant/JOMP/jomp1.0b.jar

          here my downloaded file is in /home/nishant/JOMP folder. you have to write your own location of downloaded file.


Step4: create a file and write the hello world code as given below. store file with extension .jomp

=============================================

import jomp.runtime.*;
public class Hello {
public static void main (String argv[]) throws Exception
{
int threadID;
//omp parallel private(threadID)
{
threadID = OMP.getThreadNum();
System.out.println("Hello from " + threadID);
}
}
}
=============================================
as class name is Hello ==>store file as===> HELLO.jomp



Step5: Compile the jomp directives first with following command


[nishant@ckpcet JOMP]$ java   jomp.compiler.Jomp   Hello

remember,just give file name without extension


Step6: Compile java file javac

[nishant@ckpcet JOMP]$javac Hello.java


Step7: Execute java file

[nishant@ckpcet JOMP]$ java -Djomp.threads=n Hello

set the no of threads with value of n.

Congratulation for First JOMP application