Click here to Skip to main content
15,867,568 members
Articles / Operating Systems / Windows
Article

Getting started with a SourceForge project

Rate me:
Please Sign up or sign in to vote.
4.87/5 (59 votes)
29 Jun 200320 min read 114.9K   88   26
A brief overview of how to use some of the tools and services that come into play with a SourceForge OSS project

Introduction

This will briefly walk through some of my experiences with working on projects hosted by SourceForge, and how to use the various tools that SourceForge puts at your disposal, in addition to a brief overview of how to use some of the more frequently used command line tools.

SourceForge offers free hosting for any project that complies with an Open Source License (i.e. GPL, LGPL, BSD[ my favorite], Public Domain, Artistic, MIT, etc). To create a new project you simply register at SourceForge and then submit a new project request. Most projects are approved immediately, and you'll typically get an email notifying you of the approval in ~ 24 hours (more if you start the registration on the weekend). Once you have gotten approval, it takes another 6-12 hours for all the services to go "live" for your project. 

Once you have your project, you now have a wide number of services available to you - this is part of what makes using SourceForge so cool! Among these services are:

  • CVS hosting
  • Web hosting 
  • Database support via MySQL (each project is allowed a single database, with as many tables as they want).
  • Bug tracking
  • Feature Request tracking
  • Task tracking
  • File release support for tracking and organizing the various files you want to distribute to the public.
  • A Documentation Manager
  • Compiler farm access for building source code on alternate compilers (if this matter)
  • Project shell access

Much of this is fairly straight forward to use, however some things are are bit more tricky if you are working on Win32 based system. Many of the tools that SourceForge expect you to use are traditionally Unix/Linux based command line tools, and as such may be a bit weird to get used to. For example, all access to SourceForge shells and CVS repositories are done via using the Secure SHell tool (SSH), a kind of super encrypted telnet replacement, and Secure CoPy (SCP), again, an encrypted replacement for FTP access. Getting this to work on Win32 may take a bit of tweaking.

SSH and SCP

First off, before you do anything (assuming you'll be doing most, if not all, of your work from a Win32 system), get Cygwin. This provides with a whole host of incredibly useful command line tools that makes working with SourceForge a whole lot easier. In addition, the setup program for Cygwin will automatically configure your system correctly and takes a lot of drudgery out of setting things up. The key tools (from the perspective of this article) Cygwin provides are:

  • ssh
  • scp
  • cvs
  • make
  • an entire unix compatibility layer that can help to use other unix based programs
  • a MUCH enhanced command line shell (called bash, or Bourne Again SHell ) that simply beats the pants off of using the default cmd.exe that comes with Windows. 

The Cygwin installer will allow you to pick and choose what it installs - my recommendation is to just use the defaults for now, you can always run it again and pick up extra programs later.

Once thing to note - much of this article describes techniques using command line tools. While there may be GUI tool equivalents (WinCVS for cvs, Putty for ssh) for some of the command line tools, my advice is to just suck it up, don't fight it, relax, and learn the command line! There are many many things you can do with it, it's incredibly powerful, and you'll be glad you did later on. It may be frustrating at first, but (IMHO) it is well worth the effort, especially if your project starts to scale. In addition many of these same tools/techniques are used on other OS's besides Win32 so you'll be gaining some valuable experience there.

Once Cygwin is installed, let's confirm you can access your SourceForge project's shell account. You'll need to be able to do this to start adding web content. So start up the Cygwin bash shell via Start > Programs > Cygwin > Cygwin Bash Shell.

Then type (we'll assume our SourceForge username is Neo):

ssh -l neo shell.sourceforge.net

If this is your first time, you'll see something like:

The authenticity of host 'shell.sourceforge.net (66.35.250.207)' can't be established.
DSA key fingerprint is 02:ab:7c:aa:49:ed:0b:a8:50:13:10:c2:3e:92:0f:42.
Are you sure you want to continue connecting (yes/no)?

Type yes, and then you'll be prompted for your password. Enter it and hit return.

You should see a whole text banner scroll by and finally end up with a prompt like so:

neo@sc8-pr-shell1:~$

At this point you have logged in ! If you don't reach this, then you may have network issues, such as a fire wall that won't let you through, etc. 

Once in, you can access your home directory and work with things there, or, you can move to you're projects directory. Projects are organized (at least for now, SourceForge may change this later) under /home/groups/<first letter of you project name>/<first two letters of your project name>/<project name>. For example, the VCF project is under /home/groups/v/vc/vcf. The root directory for your web site is htdocs in your project's root directory (again, using the VCF example, the VCF website is in /home/groups/v/vc/vcf/htdocs ). 

SSH is used for anything that you would have used telnet for, and is much, much more secure. SCP, on the other hand, is the secure equivalent of FTP. With SCP you can copy files to and from another remote location, using the same encryption techniques that SSH uses. So, to copy a file to your home directory you would type (again, assuming neo is your username):

scp some_file.txt neo@shell.sf.net:/home/users/n/ne/neo/ 

hit enter, and you'll be prompted for your password, enter that and your file will be copied over.

likewise to copy a file from a remote location, say from your project's website to your current directory:

scp neo@shell.sf.net:/home/groups/v/vc/vcf/htdocs/index.html ./

If you have a lot of files to upload, consider using tar and gzip to create a compressed archive and upload the archive using scp, and then, using ssh, uncompress and extract the archive in your projects shell account. For example, lets say we want to grab all the html files in our current directory:

tar -cf myhtmlfiles.tar *.html
gzip myhtmlfiles.tar
scp myhtmlfiles.tar.gz neo@shell.sf.net:/home.groups/v/vc/vcf/htdocs

This created a tar file, compressed it with gzip, and then copied it up to our web directory.

ssh -l neo shell.sf.net
{enter your password as normal}
cd /home/groups/v/vc/vcf/htdocs
tar -zxf  myhtmlfiles.tar.gz
rm myhtmlfiles.tar.gz
exit

This logged us into the project shell account, we went to the projects web directory and extracted all the html files we previously uploaded. We then removed the myhtmlfiles.tar.gz file, and then logged out (the exit command).

CVS

OK so we have the basics of downloading/uploading files around our SourceForge project. Now we need to get a feel for keeping track of our source code. For that we use CVS, which is the version control system that SourceForge provides. Traditionally speaking, CVS has always been a command line tool. Recently, there have been a number of projects to provide some sort of GUI for it on platforms like Windows. However, due to the fact that SourceForge encrypts their access to their CVS servers using SSH, some of the CVS GUI tools do not work very well (at least in my experience, if someone knows better, I'd be happy to hear about it), so you're better off simply learning the command line tool, which frankly is not that hard.

Before we start, though we have to ensure that CVS understands how to connect using SSH. Make sure you have the following environment variables set up (note, I usually set them up under User variables, however they should work equyally well as System variables).

VariableValue
CVS_RSHssh
CVSROOT:ext:<username>@cvs.<project name>.sf.net:/cvsroot<project name>

Where <username> is your SourceForge username, and <project name> is the project's unix name - you'll specify it when you register the project with SourceForge.  As an example, lets say we registered as ourself as "agent_smith" and our projects name was "The Burly Man", and our unix name was "burlyman". Our CVSROOT would need to read:

:ext:agent_smith@cvs.burlyman.sf.net:/cvsroot/burlyman

Getting this right is crucial to being able to connect with read/write access to your projects CVS repository. This also saves you from having to type this all out using the command line for each cvs command.

In general using CVS is not too difficult. The most common things you'll want to do is check out files, check them in, update files, and perhaps check their status. You can also do things like merging, tagging, etc. For a great manual, check out CVS Home

Once you have your initial source files ready you have a choice: you can add them manually, or use the import command of cvs to import the whole directory (and any subdirectories) in one shot. Simply adding the files/directories gives you greater control, but import is simpler. However, if you do import, note that it will attempt to put everything into source control - so if you have just compiled your code and you have all sorts of .obj/.lib/.pdb etc, lying around those will get added to the repository as well as all your source code! So, before using import (if you choose to use it) be careful to clean up the subdirectories of any files that you don't want to be checked in. Also give some thought to how your directories are laid out and named. Most CVS servers run on Unix/Linux OS's and their files systems are case sensitive. Obviously, on Win32, this is not the case, so it is easy to get careless and accidentally have a directory name "Foo" that you meant to name "foo". This will in turn cause some shell scripts to fail mysteriously since the script may be expecting the directory to be in lower case ("foo") when it is in fact in upper case ("Foo"). While CVS says you can remove directories, in reality it doesn't seem to work. So if you do import/add a directory with the wrong name, you'll need to open an incident with SourceForge and ask one of the admins to rename the directory for you (this usually takes 2-3 days to happen, at least in my experience).

Lets look at some examples (note that cvs usage is always cvs <command> [options] [filename] )

Importing:

cvs import -m "some message" foobar  Main start 

the -m is the message to use for this initial import, foobar is the name of the top level directory that will be created in the CVS repository (note that this can be totally different than the directory you're currently in), Main represents a vendor tag (it can be anything), and start is the release tag (again it can be anything, see here for more info). Once you hit enter, you'll be prompted for a password, and then CVS will start at the current directory and simply import an files and sub directories it finds into the repository. 

Adding files:

cvs add foobar.h foobar.cpp

Adds foobar.h and foobar.cpp - note, again, this is case sensitive! 

Adding directories:

cvs add Widgets/
cd Widgets
cvs add *.h *.cpp

This adds the Widget directory, then cd's to the Widget directory and adds all the .cpp and .h files in it. As far as I know there is no way to add a directory and all of it's files in one step.

Adding binary files:

cvs add -kb FooAppIcon.bmp FooDocIcon.png

This will add the FooAppIcon.bmp and FooDocIcon.png files and tell the cvs server that they are binary files. This is important because by default CVS assumes everything is text files and can be merged. By telling CVS we are dealing with binary files (the -kb options) it will not attempt to merge the files, nor will attempt to perform CR/LF translations.

Checking in files:

cvs ci foo.h

This will check in the file foo.h. CVS will automatically attempt to open a VI session where you can edit a check in message for the file(s). Using VI is a bit weird, please see here for more info. Once you save your changes to the file opened in VI, CVS will continue it's check in process. Note: if you make no changes to the message file in VI, cvs will prompt you, warning you that no message was entered do want to retry and edit the message, continue, or abort the check in.

Checking in directories:

cvs ci Src/ Include/

This will check in the Src and Include directories (and any subdirectories) and all files in them. 

If you are in the top level directory for your project, you can check in everything that has changed at once by:

cvs ci

This will check in everything in the current dir and any sub directories.

 

Updating you code:

cvs update

This will update any existing files, merging the changes if possible. If new files have been added then the update will create new files. It will not create any new directories!

Updating you code (and adding new directories):

cvs update -d

This will update any existing files, merging the changes if possible. If new files have been added then the update will create new files, and if new directories have been added it will create them as well. 

This is important because if someone checks in a new directory (from the top level directory), say Baz and two new files, say Baz/Bar.h and Baz/Bar.cpp, and you simple use cvs update, you will not get the new files Baz/Bar.h and Baz/Bar.cpp. Instead you need to use cvs update -d.

That's the basics of CVS. For more information, especially regarding tagging and merging/branching see the CVS home site.

Bug tracking

SourceForge provides you with a free bug tracker, which allows you to pretty easily keep track of problems with your project. As an admin you can create categories for the list as well as modify the state of any bug in the list.  While it is not as detailed as some bug trackers out there, it certainly beats nothing at all or trying to cobble something together yourself. As an admin, you can customize the "submit new" page a bit, not nearly as much as would like, but, again, it's better than nothing. Anyone can enter bugs (unless you turn off anonymous posting), so it is relatively easy to allow your users to submit bugs here, without forcing them to become SourceForge members.

The Tracker

Partly related to the Bug tracker, is the Tracker in general, which is the engine which allows you to create/monitor things like Bugs, Support Requests,  Patches, and Feature Requests. Each section allows you to add new categories, adjust user privileges, add new groups (I've never used this one) and update general preferences (like where an email notice is sent whenever something changes or is added). In general it's quite useful to use, though I notice that may projects don't seem to use much more than the Bug tracker section.

Tasks

The tasks section allows you to keep track of who's doing what when. I really like the idea of this, but I wish it had more advanced features (like charting, for example). However, once again, it's better than nothing, and I am amazed that all this is free!

You organize your tasks into "Subprojects", which you can then browse and/or edit or add tasks to. When you add a task you can control what user is assigned it, when it should be done, what percentage of it is done, notes on the task, and dependencies of the task.

I think that this is well worth using, and if you tie in the task id with your CVS check in notes, it can make things a bit easier to track down what going on in your project (particularly if you have a large project).

File Releases

SourceForge provides your project with a fairly simple to use file release manager, which allows you to provide simple links to various project files, such as installers, patches, resources, documentation, etc. SourceForge doesn't put too many limits on how much data you can put out or how long you keep it. Some projects have file release histories going back over 3 years, and have hundreds and hundreds of megs of data as a result. 

As a project admin, you can go to your project admin section and from there get to the admin interface for file releases.  Releases are divided into two main sections, packages and releases themselves. A Package is simply a logical grouping for a series of release files and can be named anything you want. File releases are simply a name that then gets associated with the release notes you provide, as well as a link to the actual release file. Each release can have 0 or more files associated with it.

Once you have a file (or files) ready to release, you use the SourceForge File Release interface to prep for it, and then upload the file via ftp to SF's temporary upload site. From there you'll select the file in the File Release interface, fill out some info on it (such as what type of file is it, a Win32 exe, a compressed collection of source files, etc), and then SourceForge will take care of the rest. 

You can upload the files anonymously via ftp to upload.sourceforge.net in the incoming directory. 

Documentation

SourceForge provides a documentation manager, but almost no one uses it (other that SourceForge). Unfortunately it has some rather severe limits as to what you can manage with it. I have tried a couple of different things, and have currently settled on DocBook and doxygen for my purposes. I would highly recommend them to others as well. DocBook allows you to have a single documentation source and then using XSLT it can transform that source into various output formats such as PDF, HTML, HTML Help (complete with the project, index and search files ready to go), TeX, and others. Hopefully I'll be able to write an article on how to use DocBook (again with a slant towards getting it to work on Win32 systems) in the near future. During the transformation, DocBook takes care of all sorts of things automatically, such as auto indexing stuff, auto indenting the various sections, allowing you to cross reference things, etc. 

Miscellaneous tricks

Updating files, or touching yourself...

Touch is a little utility you can use to update a file timestamp to the current time. One use might be if you need to force a file to be up to date to get a makefile to pick it up. 

touch foo.h

Another use is to create a new file. For example, running touch Blah.cpp will create the file Blah.cpp if it didn't already exist.

Patching and Diff

Once your project is up and running, you may find that people will send you a patch. Code patches are reasonably common in the OSS world (or at least I have seen a fair amount their usage), but not so much in the Win32 world (again, in my experience). 

The diff program will output the differences between 2 files in a standard text format (the same format that is used by CVS to merge and store changes). For example let create a file called foo.h that looks like so:

class Foo {
    foo();
    void doit() {
        int x = 0;
        int y = x * sin(0.456);
    }
};

now modify it like so, and save it as foo.h.1

class Foo {
    Foo();
    void doit() {
        int x = 0;
        int y = x * sin(0.456);
    }
};

All we did was add the capitalization to the Foo constructor. Now run diff to see the difference between the two files. 

diff -u foo.h foo.h.1
--- foo.h       2003-06-30 16:50:19.000000000 -0400
+++ foo.h.1     2003-06-30 16:33:35.000000000 -0400
@@ -1,5 +1,5 @@
 class Foo {
-       foo();
+       Foo();
        void doit() {
                int x = 0;
                int y = x * sin(0.456);

What you're seeing is the output of the diff program. Using the -u argument alters the output format and makes diff use the unified output format, which is what other tools use as well. We can dump this to a file like so:

diff -u foo.h foo.h.1 > foo.h.diff

We now have the differences stored in file. Which in turn brings us to patch...

Patch allows you to take a patch file and apply it to a source file. So we could take our foo.h.diff, apply it to our original foo.h, and have the changes applied, in this case the Foo() constructor fix. To apply the changes, simply use

patch foo.h foo.h.diff

The point of all this is that you can now use patches sent from other developers. For example, the GTK project allows outside developers (who typically do not have cvs write access) to submit patches, which are also required to be gzipped as well, which the GTK development team can then apply at a later date, if the patch is accepted. You can make use of a similar system in your project if this interests you.

 Getting further help on command line programs

At some point in using some of these programs, you'll want to get further information  about how to use them and how they work. Traditionally this was done by typing man <program name>, which in turn would display the help as text, that was stored in text files called man pages. However man pages are for old men! There is a newer help tool called info that allows you to navigate the help and even allows you to position the cursor over certain "links" and be taken to the referenced section. Most, if not all, of the Cygwin command line documentation is available via info. So if you had a question on how to use the diff tool, just type:

info diff

Which immediately takes you to something like this:

File: diff.info,  Node: Invoking diff,  Next: Invoking diff3,  Prev: Invoking c\
mp,  Up: Top

Invoking `diff'
***************

   The format for running the `diff' command is:

     diff OPTIONS... FILES...

   In the simplest case, two file names FROM-FILE and TO-FILE are
given, and `diff' compares the contents of FROM-FILE and TO-FILE.  A
file name of `-' stands for text read from the standard input.  As a
special case, `diff - -' compares a copy of standard input to itself.

   If one file is a directory and the other is not, `diff' compares the
file in the directory whose name is that of the non-directory.  The
non-directory file must not be `-'.

   If two file names are given and both are directories, `diff'
compares corresponding files in both directories, in alphabetical
order; this comparison is not recursive unless the `-r' or
`--recursive' option is given.  `diff' never compares the actual

From here you can navigate around using the arrow keys, using the "n" key for the next topic, "p" for the previous topic, and "u" to navigate to the parent topic. There are other keys that work as well but this will get you started.

Info works on most systems now, I think any modern Linux distribution will have it installed, and probably Solaris as well. So if you're logged into the SourceForge shell and don't know how to use a command try info for help.

Conclusion

Much of this info is scattered around the SourceForge site in various places, and hopefully it will help you if you ever decide to use SF's services. If there are things that people feel are missing and would explained in greater detail, just post comments at the bottom and I'll try and incorporate the requests by updating the article as time permits.

Using VI

VI is a command line text editor. It is a bit weird to get used, and while some people like it, I do not. However it is important that you are at least able to do a little bit of editing in it, because some applications use it for editing text messages (CVS), and there may be times on a system where it is the only functioning editor (shudder...).

VI works in modes - there's a insert mode and a command mode (for lack of a better term). It starts out in command mode. If you open a a file with (say vi foo.h) you can exit, assuming no changes were made, by hitting ":" key and then typing "q" (you should see the text show at the lower left hand corner of your terminal window as ":q"), hit enter and vi should quit. I you made changes, ":qw" quits and writes the changes out. To quit without saving use ":q!".  To make sure you're in command mode hit the "Esc" key (some version of VI also switch to command mode if the arrow keys are used as well) prior to typing.

To enter some text in VI you need to enter insert mode. Use the arrow keys to navigate to where you want to start typing and then hit either "a" (to append to the text) or "i" to insert text. Typically with a new file you'll hit "a". Now starting typing as normal to enter text. Note that on some versions/systems hitting the arrow keys will take you out of insert mode and put you in command mode (you'll no because no text will be entered onto the screen anymore), so be careful. In fact, using VI is a bit like attempting to use a nuclear powered chainsaw for watch repair - its an extremely powerful and blunt edged tool - you've been warned.

To quit entering text hit the "Esc" key. To delete text, go to command mode, use the arrow keys to navigate to the character and hit "x". To delete a whole line use "dd". That's pretty much the extent of my knowledge, so have fun!

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
Software Developer (Senior)
United States United States
Currently working on the Visual Component Framework, a really cool C++ framework. Currently the VCF has millions upon millions upon billions of Users. If I make anymore money from it I'll have to buy my own country.

Comments and Discussions

 
GeneralNice one Pin
Member 1265872928-Jul-16 6:30
Member 1265872928-Jul-16 6:30 
Questionssh: command not found ??? Pin
liuty200610-Nov-04 15:25
liuty200610-Nov-04 15:25 
AnswerRe: ssh: command not found ??? Pin
Jim Crafton15-Nov-04 14:50
Jim Crafton15-Nov-04 14:50 
GeneralSmall change Pin
benjymous20-Oct-04 2:08
benjymous20-Oct-04 2:08 
GeneralExcellent! Pin
Marc Clifton1-Jul-03 0:17
mvaMarc Clifton1-Jul-03 0:17 
GeneralRe: Excellent! Pin
PFunky1-Jul-03 12:06
PFunky1-Jul-03 12:06 
GeneralEasier CVS Pin
NiceGuyUK30-Jun-03 22:16
NiceGuyUK30-Jun-03 22:16 
GeneralRe: Easier CVS Pin
Jim Crafton1-Jul-03 3:06
Jim Crafton1-Jul-03 3:06 
GeneralRe: Easier CVS Pin
NiceGuyUK1-Jul-03 3:15
NiceGuyUK1-Jul-03 3:15 
One of the putty tools is called PLink which handles the SSH keys and passwords required. Tortoise CVS is able to work with PLink. I can't remember where I found the exact details, but it explains this in the docs on the TortoiseCVS site somewhere.

Rob.

[UPDATE] - info on this at http://www.tortoisecvs.org/faq.html#sshkeys
GeneralRe: Easier CVS Pin
Steven Behnke1-Jul-03 7:59
Steven Behnke1-Jul-03 7:59 
GeneralRe: Easier CVS Pin
NiceGuyUK2-Jul-03 22:23
NiceGuyUK2-Jul-03 22:23 
GeneralI can't believe it Pin
Jim Crafton30-Jun-03 16:48
Jim Crafton30-Jun-03 16:48 
GeneralRe: I can't believe it Pin
Ryan Binns30-Jun-03 16:59
Ryan Binns30-Jun-03 16:59 
GeneralRe: I can't believe it Pin
Jim Crafton30-Jun-03 17:58
Jim Crafton30-Jun-03 17:58 
GeneralRe: I can't believe it Pin
Abbas_Riazi30-Jun-03 18:53
professionalAbbas_Riazi30-Jun-03 18:53 
GeneralRe: I can't believe it Pin
Uwe Keim30-Jun-03 19:09
sitebuilderUwe Keim30-Jun-03 19:09 
GeneralRe: I can't believe it Pin
eXplodus30-Jun-03 21:41
eXplodus30-Jun-03 21:41 
GeneralRe: I can't believe it Pin
Uwe Keim30-Jun-03 19:09
sitebuilderUwe Keim30-Jun-03 19:09 
GeneralRe: I can't believe it Pin
Wesner Moise30-Jun-03 22:11
Wesner Moise30-Jun-03 22:11 
GeneralRe: I can't believe it Pin
Marc Clifton1-Jul-03 0:16
mvaMarc Clifton1-Jul-03 0:16 
GeneralRe: I can't believe it Pin
Jim Crafton1-Jul-03 3:01
Jim Crafton1-Jul-03 3:01 
GeneralRe: I can't believe it Pin
Pavel0073-Jul-03 11:53
Pavel0073-Jul-03 11:53 
GeneralRe: I can't believe it Pin
Marc Clifton3-Jul-03 12:04
mvaMarc Clifton3-Jul-03 12:04 
GeneralGreat Work! Cheers Pin
Anonymous30-Jun-03 16:43
Anonymous30-Jun-03 16:43 
GeneralGreat Job, Jim Pin
Nemanja Trifunovic30-Jun-03 12:15
Nemanja Trifunovic30-Jun-03 12:15 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.