Click here to Skip to main content
15,867,568 members
Articles / Mobile Apps / Windows Mobile
Article

A C# component to encode and decode yEnc data

Rate me:
Please Sign up or sign in to vote.
4.54/5 (17 votes)
12 Apr 2004CPOL4 min read 104.7K   3K   39   18
A fully tested component capable of encoding and decoding yEnc data.

Sample Image - yenc.gif

Introduction

This is an implementation of the yEnc algorithm, as described at http://www.yenc.org/ . yEnc is not an official standard, but it is nonetheless a very popular encoding method on binary newsgroups. As algorithms go, yEnc is very simple. It uses 8-bit characters to encode binary data. Since binary data is usually stored as 8-bit bytes, it does not have to accomplish too much :)

yEnc's popularity is due to the fact that it uses a full byte to encode the data, whereas other methods use only 7-bits. This makes messages encoded with yEnc smaller by a factor of 33-40%, according to the website. Smaller means quicker to upload and download, which is important when dealing with large binary files. It has additional benefits as well, in the form of an optional CRC32 check.

My implementation is interesting from 2 points of view:

  1. It is the only open-source one written in C#
  2. It is implemented as a cryptographic transform - more on that later

Some Info on yEnc

There are some peculiarities of newsgroup messages:

  • messages must be broken into lines, max around 1000 characters
  • some characters have meaning, and as such need to be escaped out

The current yEnc algorithm escapes out CR, LF and the NULL character by default. However, individual encoders are free to escape other characters as they wish. Lines are broken at 128 characters, or 256 characters, by convention. Other line lengths are supported.

yEnc data begins with a =ybegin tag at the start of a line. The tag has additional attributes that specify the number of bytes to expect, as well as the name of the file and the length of the lines. Multipart messages are supported. The data ends with a line starting with =yend. The reason for the "=y" is that, due to the nature of the algorithm, it could never occur naturally as part of the data.

Using the code

My implementation of the algorithm deals purely with encoding and decoding the data, not parsing of messages, or even parsing of the yEnc headers and footers. To me, that is a separate challenge, which I'll leave to someone else.

Initially, I started coding the encoder as an implementation of System.Text.Encoder. However, I soon realized that, although I could read the data as text, I was really dealing with bytes. Probably, that should have been obvious to me from the beginning, but sometimes it takes a while :( Eventually, I decided it would work best as an implementation of ICryptoTransform. This is not to imply that it is a cryptographic algorithm, just that it transforms data in similar ways - the size of the input data does not necessarily match the size of the output data. Microsoft chose to implement the Base64 transformation objects in a similar way.

The benefit is that you can use the objects together with a CryptoStream object, which is a fairly easy interface to use, and automatically adds support for streams. I'll stress again though, that this is not an encryption technique - I am just making use of existing Framework objects and interfaces to add power to my objects.

To encode some yEnc data, your code might look like this:

C#
MemoryStream ms = new MemoryStream();  <BR>  //this could be any stream we want to write to

YEncEncoder encoder = new YEncEncoder();
CryptoStream cs = new CryptoStream(ms, encoder, CryptoStreamMode.Write);
StreamWriter w = new StreamWriter(cs);
w.Write("Test string");

//make sure everything is written out
w.Flush();
cs.Flush();

To decode it again, the code might continue:

C#
//reading back from the memorystream
ms.Position = 0;        
YEncDecoder decoder = new YEncDecoder();
CryptoStream cs2 = new CryptoStream(ms, decoder, CryptoStreamMode.Read);
StreamReader r = new StreamReader(cs2);
string finalText = r.ReadToEnd();

This is pretty standard code that you might write if you were encrypting your data, the only difference being that we are using the yEncEncoder and yEncDecoder instead of a system-supplied encryption algorithm.

Points of Interest

I have made use of Phil Bolduc's implementation of the CRC32 algorithm, found at http://www.codeproject.com/csharp/crc32_dotnet.asp . Unfortunately, there were some bugs in that that consumed a significant amount of my time. I had to make some modifications to make it work 100%. Other than that, the code is an original work of my own, not based off of any other implementations. You are free to use it for whatever purpose you may desire, as long as you attribute it to me in the code comments.

The dowloadable code includes a lot of NUnit tests, which test things to a point where I am comfortable that everything works. They should make the code easy to expand on for anyone who wants to add functionality.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Web Developer
United States United States
Steve is a software developer working in Minneapolis, MN.

Comments and Discussions

 
GeneralMy vote of 1 Pin
firepacket14-Dec-20 8:28
firepacket14-Dec-20 8:28 
Bugtotal crap. garbage. broken Pin
firepacket14-Dec-20 9:07
firepacket14-Dec-20 9:07 
GeneralC# unchecked keyword on Line 173 of Decoder.cs Pin
davejuk22-Oct-09 6:32
davejuk22-Oct-09 6:32 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pin
Steven Campbell24-Oct-09 18:40
Steven Campbell24-Oct-09 18:40 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pin
davejuk25-Oct-09 3:40
davejuk25-Oct-09 3:40 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pin
davejuk25-Oct-09 9:42
davejuk25-Oct-09 9:42 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pin
davejuk26-Oct-09 7:37
davejuk26-Oct-09 7:37 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pin
Harmen29-Nov-09 8:42
Harmen29-Nov-09 8:42 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pin
Member 259847821-Dec-09 10:54
Member 259847821-Dec-09 10:54 
The only thing i realized when i finally got this to work with my nntp stream(issues due to text encoding) was that it doesnt want any of the =ybegin lines, or end= lines, just the data itself.

If i attempted to run a file through with those still it would decode them into a bunch of garbage at the top/bottom of the text file i was creating, would probably totally corrupt any other executable type of file.
GeneralAwesome try... Pin
alexn29-Nov-06 17:52
alexn29-Nov-06 17:52 
GeneralRe: Awesome try... Pin
alexn6-Dec-06 12:20
alexn6-Dec-06 12:20 
GeneralRe: Awesome try... Pin
Binindex21-Jan-07 11:39
Binindex21-Jan-07 11:39 
GeneralRe: Awesome try... Pin
fdg sdfg8-Jul-09 4:02
fdg sdfg8-Jul-09 4:02 
GeneralRe: Awesome try... Pin
Binindex8-Jul-09 5:34
Binindex8-Jul-09 5:34 
GeneralDecoding giving me a hard time Pin
Carl Mercier22-Oct-04 9:54
Carl Mercier22-Oct-04 9:54 
GeneralRe: Decoding giving me a hard time Pin
Steven Campbell22-Oct-04 17:09
Steven Campbell22-Oct-04 17:09 
GeneralThanks... Pin
JaseNet9-Apr-04 18:36
JaseNet9-Apr-04 18:36 
GeneralRe: Thanks... Pin
Steven Campbell10-Apr-04 17:35
Steven Campbell10-Apr-04 17:35 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.