Creating local yum repository and using it

Sometimes it is needed to create a local yum repository when you want yum to download packages from a local repository instead of the main repository. It is very much needed when you don't want to commit your changes to a branch which is being developed by other teams. This approach is very useful when some of the packages are available in both the repositories, but you only want yum to download from your local repository for those packages.

Step-by-step guide

Before following the steps, you should have your jenkins setup in a VM (i.e. avoid the main jenkins server). Perform the steps in the same vm.

• Creating the yum repo

• Install createrepo: "yum install createrepo"
• Create the repository directory: "mkdir -p /opt/repo/RPMS/noarch"
• [Follow this page for more info: http://www.techrepublic.com/blog/linux-and-open-source/create-your-own-yum-repository/609/ ]
• To be able to use the repository through http, create a file /etc/httpd/conf.d/repo.conf:

                     Alias /repo "/opt/repo/RPMS/"

                     <Directory "/opt/repo/RPMS/">
                     Options +Indexes
                     FileETag MTime Size
                     AllowOverride None
                     Order allow,deny
                     Allow from all
                     </Directory>

• Restart httpd: sudo service httpd restart
• Now you'll be able to access the repository through: http://<server_address>/repo/noarch/

• [This step is only required if you want jenkins to build rpm and automatically put in your repository. If you put your rpms manually then skip this step] Create the jenkins job for the specific branch in your jenkins installation

• In the post step of the job configuration, select "Run only if build succeeds" and put "execute shell":

                     if [ -f ${WORKSPACE}/target/rpm/RPMS/noarch/*\.rpm ];
                     then
                     /bin/cp -f ${WORKSPACE}/target/rpm/RPMS/noarch/*\.rpm /opt/repo/RPMS/noarch /usr/bin/createrepo /opt/repo/RPMS/noarch
                     fi

• The above will copy the rpm built rpm to the yum repository directory.

• In the target server (i.e. where you want to run yum) install yum-priorities: 

                     yum install yum-priorities

          This plugin enables yum to search for packages based on priority.

• Setup the repo file like (for example /etc/yum.repos.d/my-repo.repo):

                     [main-repo]

                     name=main-repo
                     baseurl=http://<server_address_of_main_repo>/repo/LATEST/
                     gpgcheck=0
                     enabled=1
                     metadata_expire=1
                     priority=2

                     [my-repo]
                     name=my-repo
                     baseurl=http://<server_address>/repo/noarch/
                     gpgcheck=0
                     enabled=1
                     metadata_expire=1
                     priority=1

          The above tells yum to first check in my-repo repo, if not found then to go to main-repo (note the priority=1).

Now you are ready to use your yum repository. As soon as the rpm is built by jenkins, it can be fetched using yum.

Don't mess with /System/Library/Frameworks/JavaVM.framework/Versions/Current in Mac OS X

I had both JDK1.6 and JDK1.7 installed in OS X 10.9 (Mavericks).  I was trying to force java to use default JDK version to 1.6. I thought its going to be fine if I change the soft link:

/System/Library/Frameworks/JavaVM.framework/Versions/Current -> A

to this:

/System/Library/Frameworks/JavaVM.framework/Versions/Current -> CurrentJDK

where CurrentJDK points to /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents (the JDK 1.6).

But after doing so, I got this error when trying to compile an application that depended on AWT libraries:

java[5617:d07] Apple AWT Startup Exception : -[__NSCFString appendString:]: nil argument
java[5642:d07] Apple AWT Restarting Native Event Thread

Also "/usr/libexec/java_home" (which gives the java_home) was also giving "command not found" error.

After doing some googling, I found that I shouldn't mess with /System/Library/Frameworks/JavaVM.framework/Versions/Current. So it should be kept as it was (pointed to A). Rather just change the link of /usr/bin/java to /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java:

sudo rm /usr/bin/java
sudo ln -s /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java /usr/bin/java

Then /usr/libexec/java_home should automatically point to /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home.

Now you should set your $JAVA_HOME in your ~/.bash_profile file like this:

export JAVA_HOME=$(/usr/libexec/java_home)

Then source the ~/.bash_profile file:

source ~/.bash_profile

You're all set now with your preferred java version.

Getting started with GIT basics

Previously I've blogged about making a local SVN repository. This time I'm going to talk about GIT, another version control system. For many reasons, GIT has advantages over SVN (which I'm going to blog about some other time). It is also a little bit complex compared to SVN. For example, if you modify a file in SVN, then that modification persists in your local machine even without committing. Then later on you can commit that modification so the changes are also applied to the main repository. But it's a little different in GIT. In GIT you have the repository in your local machine (whereas in SVN your local copy acts just like a client). After modifying a file in GIT repository, you need to commit that change just to make sure that the changes persist in your local repository for the current branch (not in the original repository). If you modify a file but don't commit, then you can't push it to the original repository. If you want your modification to be available in other branches, then you'll need to merge the two branches. At last, you'll need to "push" your changes to the original repository.

There are plenty of graphical tools to manage GIT operations, you'll never have to think about the command line if you use those tools. But its good to know what commands are working in the background. I really like SourceTree as a git manager. If you use Eclipse as your IDE, then you can also install the EGit plugin which provides very nice GIT Team management.

Here are some very basic GIT commands to get you started with it, provided that you already have access to a GIT repository somewhere and you've installed GIT in your local machine. You might need to setup a ssh-key based authentication from your computer to the original repository (depending on the settings of the repository).

Let's say the remote repository exists at ssh://git-user@server01/git-test.git and you want to clone only develop branch (which is analogous to trunk in SVN) for now.

1. From command line (in windows, install the command line tool for GIT) put this command to create the local repository:

• git clone -b develop ssh://git-user@server01/git-test.git

2. It will create a new folder named git-test. Enter the folder and you can create your own branch (and you'll be switched to this new branch):

• git checkout -b rafis_branch

3. Now there are two branches in your local machine, namely develop and rafis_branch. You can see the branches with this command:

• git branch -l

You'll see something like:

develop

* rafis_branch

The * denotes the current branch you're in.

4. Modify or add a new file. Now at this point you still haven't committed your modification to any branches. So if you change the current branch without committing, you'll see the same changes in other branches (in your local repository) as well. You can check the current status with this command:

• git status

5. If you've added a new file, then use this command to track this file:

• git add new_file.txt

We still haven't committed the changes (we've merely added this file for tracking by GIT).

If you've modified a file, then the above command is not needed since the file is already tracked by GIT. commit the changes to the current branch (also needed for newly tracked file):

• git commit -am "Added new_file.txt and modified file.txt"

The changes will now be committed to the current branch. So now if you change the branch, then you can't see the modifications made. For example, if you use graphical interface to see the file, then you must be in the branch (with terminal) where you've committed the changes. If you switch to other branch, then you can't see the new file or the modified file in the graphical interface. All these information are kept in the hidden folder called .git in the repository root.

6. Now lets make the changes available in other branch as well. Lets say you've committed the changes in rafis_branch. Now you want the change to be merged in develop branch. Then first switch to the develop branch:

• git checkout develop

7. Before merging the changes from rafis_branch, make sure that you're upto date with the origin in this branch. So you'll need to pull from the origin develop branch:

• git pull origin develop 

This will pull any new changes made in the develop branch in the origin repository. This is required to make sure you don't put any conflict in the origin. Rather merge in the local repository, fix conflicts if any, then push to the origin repository when you're convinced that there'll be not conflicts in the files.

8. Now you can merge the changes of rafis_branch to develop branch:

• git merge rafis_branch

9. After merging and fixing any conflicts in the files, now its time to push your changes to the develop branch of the origin:

• git push origin develop

Voila, you've just pushed your first GIT commit to the origin repository. Now when someone pulls from the origin repository, they'll get the changes you made.

10. You can continue working on your new branch (don't forget to switch to that branch "git checkout rafis_branch"). If you think that you no longer need the branch you created in your local machine, then you can delete it (and can create again later on):

• git branch -d rafis_branch

Now that branch doesn't exist anymore. So the command "git branch -l" will not show that branch now.

Setting the locale in UNIX environment

Recently I upgraded from OS X 10.8 (Mountain Lion) to 10.9 (Mavericks), and noticed that my locale is set to (you can check this with command "locale"):

LANG="C"
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL="C"

You'll experience some encoding problems with this locale. Also when you want to remote login to a Redhat machine, you'll encounter some warning. To set the locale to your preferred one (for me its "en_US.UTF-8"), you'll need to add these two lines in your ~/.bash_profile file:

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

That's all, now you can open a new session (or just use this command within the running session: "source ~/.bash_profile" ) with your terminal and check the locale again. It should be now like this:

LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

Installing and using VirtualBox in Fedora

Often we have to work in more than one platform simultaneously while developing something (or for any other work as well). And it is very disturbing to switch to different operating systems from time to time.

We have a very good solution to that. We can use VirtualBox to install multiple guest operating systems in one host operating system. For example, lets say we want to install Windows Vista in Fedora. So our host system is Fedora and guest system is Window Vista.

1. First we'll have to download and install the latest VirtualBox in Fedora. After installing, open VirtualBox.

2. Click on 'New'. Then press next. It will ask you to give a name of the new operating system, you can give any name for this. Below VirtualBox will ask for the operating system type and version (in this case, type is Microsoft Windows and version is Windows Vista).

3. Then you'll be asked to specify the amount of memory (RAM) to be used for this guest OS. Give it as you like (better to give atleast 512mb for Vista). Note that, your total RAM will be divided for the host and guest OS when you run guest OS.

4. Then it will ask to specify the hard disk space for the guest OS. When creating a new partition, there are two options. Fixed-size storage (the drive size will be fixed) and Dynamically expanding storage (the drive will start with a small size, but it will expand as needed to the maximum size specified by you).

5. Click finish button. Then start the new created operating system by pressing 'start'. It will ask for the Windows Vista cd for the first time to install. It will continue the install process normally.

6. After installation, you are ready to use your new guest OS.


Sharing files between Fedora and Windows Vista:

1. We have to download the VirtualBox guest addition. It is a cd image file, if you double click on it, it will be shown as a cd drive.

2. From the windows Vista window, click devices->Mount Cd/Dvd-rom->cd/dvd-rom image. Then select the cd image you just created by double clicking on guest addition. Now this drive will be shown in 'my computer' as a drive. Go inside it and install guest addition.

3. Shut down your guest OS. From VirtualBox settings->shared folders, select the folder you want to share between the two operating systems.

4. Now restart Windows Vista. You will find an icon of VirtualBox guest addition in the taskbar.

5. Now go to command prompt and type: net use x: \\vboxsvr\shared -p

6. If successfully completed, you will find that shared folder in 'my computer' as a drive.

Hope it will help you using multiple operating systems in your machine.

Marking duplicates in Excel/Calc

When you have a big Excel/Calc spreadsheet, you often need to see if you have duplicate entries in a column. I also wanted to find out if there's any way to mark the corresponding rows if different columns hold duplicate entries. I did not find any direct solution to this, I had to do it in a few steps.

You have to select the all the entries (click on the first element of the first row, then press ctrl+shift+End), then sort them in ascending order (go to Data->Sort, select the column you want to find the duplicate in, then press OK). Make a new column. For example, if the column that you want to search for duplicate in, is A, and the new column is E, and if the entries start from A2, then put this formula in E2: "=IF((LOWER(A2)=LOWER(A1))OR(LOWER(A2)=LOWER(A3));1;0)". Set the formula to the whole "E" column. It will show 1 if it is duplicate entry, 0 otherwise.

Now, again select the whole range of data, then go to Format->Conditional Formatting. Under Condition1, select "Formula is", and then enter this formula: "=INDIRECT("E"&ROW())=1", then select the format you want the duplicates rows to show. Thats all, now you'll easily find out the duplicate entries.

If anyone know the direct solution to do this, please tell me.......

Setting up a local SVN server in Fedora Core

hi there......this is my first blog post. So I am not being able to decide which topic I should start with. I am going to start with a very useful topic, "Setting up a local SVN server".

SVN (Subversion) is a version control system which is used to control the changes of files maintained by many people. For example, if many developers are working on a project from different locations (or even from same location), few person can work with the same file and then save it with the changes they made to that file. SVN is a very smart tool that can identify the changes made my each developer and then updates the file in server with all the changes, with no error or bug (SVN actually changes the lines in files rather than overwriting the file). It sounds very unclear first up, but you'll get used to it and understand its concept once you start using it.

Setting up SVN locally can be very useful for a office or a group of developers, so that everyone can work with the same files without having to think about the collisions.

I am going to mention the steps to setup a local SVN server below:

1. You need to have SVN installed in your system. It is installed by default in Fedora Core. If you dont have it, then you can download it using yum.

2. go to terminal and type:

'svnadmin create svn_repo'

(you can use your preferred name instead of svn_repo), It will create a folder called svn_repo in your current directory.

3. cd to the svn_repo/conf folder:

'cd svn_repo/conf/'

4. open passwd file in text editor:

'gedit passwd'

under [users] add your username and password like this:

[users]
rafi = 123456

then save the file.

5. open svnserve.conf in text editor:

'gedit svnserve.conf'

under [general], write these:

anon-access = read
auth-access = write
password-db = passwd
realm = My First Repository

6. Go to the first directory:

'cd ../..'

7. 'svnserve -r svn_repo/ -d'

8. Now you'll have to import the files you want to keep in the svn server (e.g. the source files), if you want to import trunk folder in your local server, type:

'svn import trunk/ svn://localhost/local_trunk -m "Initial creation" '

9. You can look at the svn tree: 'svnlook tree svn_repo/'

10. copy the files to a location: 'svn checkout svn://localhost/local_trunk'

It will copy the files in local_trunk folder in your home directory

11. Now change a file in local_trunk, e.g. readme, then check status:

'svn status'

(you must be in the local_trunk folder), you'll see 'M' before readme file, 'M' means modified.

12. Commit your modification to the SVN server:

svn ci WordForge/README -m "readme modified"

-m means message, you must provide a message.

13. If you want to be updated to the latest changes made by other:

'svn up'

14. How others can access the SVN server from their computer:

being root:

svn co svn+ssh://[ip of the server]/home/[user]/svn_repo/ /[path to store in client computer]

for example:

svn co svn+ssh://192.168.1.17/home/Rafi/svn_repo/ /home/ankur/Desktop/

being user:

svn co svn+ssh://user@[ip of the server]/home/[user]/wf_repo/ /[path to store in client computer]

for example:

svn co svn+ssh://Rafi@192.168.1.17/home/Rafi/wf_repo/ /home/ankur/

It will ask for the password of the user of server machine.

15. Now the user from client computer can update his source by going to the directory where he stored the files, then typing:

svn up

It will update the files to the latest version.

I hope it will be usefull for you......