C# File Handling

Array Collections File String Windows VB.NET Algorithm ASP.NET Cast Class Compression Convert Data Delegate Directive Enum Exception If Interface Keyword LINQ Loop Method .NET Number Regex Sort StringBuilder Struct Switch Time Value

Files store data. Objects stored in memory cease to exist when your program ends. Objects in files exist until deletion. File data is needed in nearly every non-trivial C# program. It is handled with the System.IO namespace and the types there.

One of the most significant sources of inefficiency is unnecessary input/output (I/O). McConnell, p. 598

StreamReader

Two of the most useful types for handling files in the C# language and .NET Framework are the StreamReader and StreamWriter types; they are described here. Often, you can achieve better performance with StreamReader and StreamWriter than with static File methods.

Program that uses ReadLine [C#]

using System.IO;

class Program
{
    static void Main()
    {
	// Read in every line in the file.
	using (StreamReader reader = new StreamReader("file.txt"))
	{
	    string line;
	    while ((line = reader.ReadLine()) != null)
	    {
		// Do something with line
		string[] parts = line.Split(',');
	    }
	}
    }
}

Result
    Lines in text file are read in and separated on commas.

File.ReadAllText

This simple program uses the File.ReadAllText method to load in the file "file.txt" on the C: volume. Then, it prints the contents of the file, which are now stored in a string object.

System.IO namespace [C#]

//
// Include this namespace for all the examples.
//
using System.IO;

Program that reads in file [C#]

using System;
using System.IO;

class Program
{
    static void Main()
    {
	string file = File.ReadAllText("C:\\file.txt");
	Console.WriteLine(file);
    }
}

Output

File contents.

File.ReadAllText benchmark. Here we want to resolve whether File.ReadAllText is efficient. To answer this, the ReadAllText method was benchmarked against StreamReader. The result was that on a 4 KB file it was almost 40% slower.

Program that uses ReadAllText and StreamReader [C#]

using System.IO;

class Program
{
    static void Main()
    {
	// Read in file with File class.
	string text1 = File.ReadAllText("file.txt");

	// Alternative: use custom StreamReader method.
	string text2 = FileTools.ReadFileString("file.txt");
    }
}

public static class FileTools
{
    public static string ReadFileString(string path)
    {
	// Use StreamReader to consume the entire text file.
	using (StreamReader reader = new StreamReader(path))
	{
	    return reader.ReadToEnd();
	}
    }
}

Benchmark results

File.ReadAllText:         155 ms
FileTools.ReadFileString: 109 ms

StreamReader helper. In some projects, it would be worthwhile to use the above ReadFileString custom static method. In a project that opens hundreds of small files, it would save 0.1 milliseconds per file.

File.ReadAllLines

Here you want to read all the lines in from a file and place them in an array. The following code reads in each line in the file "file.txt" into an array. This is efficient code; we provide performance information later on.

Program that uses ReadAllLines [C#]

using System.IO;

class Program
{
    static void Main()
    {
	// Read in every line in specified file.
	// ... This will store all lines in an array in memory,
	// ... which you may not want or need.
	string[] lines = File.ReadAllLines("file.txt");
	foreach (string line in lines)
	{
	    // Do something with line
	    if (line.Length > 80)
	    {
		// Example code
	    }
	}
    }
}

Result
    We loop over lines in the file.
    We test the length of each line.

File.ReadAllLines performance. When you read in a file with File.ReadAllLines, many strings are allocated and put into an array in a single method call. With StreamReader, however, you can allocate each string as you pass over the file by calling ReadLine. This makes StreamReader more efficient unless you need all the file data in memory at once.

Read into List. We look at a usage of the List constructed type with file handling methods. List and ArrayList are extremely useful data structures for C# programmers, as they allow object collections to rapidly expand or shrink. Here we look at how you can use LINQ to get a List of lines from a file in one line.

Program that uses ReadAllLines with List [C#]

using System.Collections.Generic;
using System.IO;
using System.Linq;

class Program
{
    static void Main()
    {
	// Read in all lines in the file,
	// ... and then convert to a List with LINQ.
	List<string> fileLines = File.ReadAllLines("file.txt").ToList();
    }
}

Count lines. Here we need to count the number of lines in a file but don't want to write lots of code to do it. Note that the example here doesn't have ideal performance characteristics. We reference the Length property on the array returned.

Program that counts lines [C#]

using System.IO;

class Program
{
    static void Main()
    {
	// Another method of counting lines in a file.
	// ... This is not the most efficient way.
	// ... It counts empty lines.
	int lineCount = File.ReadAllLines("file.txt").Length;
    }
}

Query example. Does a line containing a specific string exist in the file? Maybe you want to see if a name or location exists in a line in the file. Here we can harness the power of LINQ to find any matching line. See also the Contains method on the List type.

Program that uses LINQ on file [C#]

using System.IO;
using System.Linq;

class Program
{
    static void Main()
    {
	// One way to see if a certain string is a line
	// ... in the specified file. Uses LINQ to count elements
	// ... (matching lines), and then sets |exists| to true
	// ... if more than 0 matches were found.
	bool exists = (from line in File.ReadAllLines("file.txt")
		       where line == "Some line match"
		       select line).Count() > 0;
    }
}

File.ReadLines

In constrast to File.ReadAllLines, File.ReadLines does not read in every line immediately upon calling it. Instead, it reads lines only as they are needed. It is best used in a foreach-loop.

File.WriteAllLines

Here we look at how you can write an array to a file. When you are done with your in-memory processing, you often need to write the data to disk. Fortunately, the File class offers an excellent WriteAllLines method. It receives the file path and then the array to write. This will replace all the file contents.

Program that writes array to file [C#]

using System.IO;

class Program
{
    static void Main()
    {
	// Write a string array to a file.
	string[] stringArray = new string[]
	{
	    "cat",
	    "dog",
	    "arrow"
	};
	File.WriteAllLines("file.txt", stringArray);
    }
}

Output

cat
dog
arrow

File.WriteAllText

A very simple method to call, the File.WriteAllText method receives two arguments: the path of the output file, and the exact string contents of the text file. If you need to create a simple text file, this is an ideal method.

Program that uses File.WriteAllText [C#]

using System.IO;

class Program
{
    static void Main()
    {
	File.WriteAllText("C:\\perls.txt",
	    "Dot Net Perls");
    }
}

Output
    C:\perls.txt contains the string "Dot Net Perls"

File.AppendAllText

Here we mention a way you can append text to files in a simple method. The previous example will replace the file's contents, but for a log file or error listing, we must append to the file. Note that we could read in the file, append to that in memory, and then write it out completely again, but that's slow.

File.AppendText

The File.AppendText method returns a StreamWriter instance that you can use to append string data to the specified file. It is not covered on this site in detail because it is usually easier to use the StreamWriter constructor directly.

File.ReadAllBytes

Here we use File.ReadAllBytes to read in an image, PNG, to memory. One example usage of this sample is to cache an image in memory for performance. This works very well and greatly outperforms reading in the image each time.

Program that caches binary file [C#]

static class ImageCache
{
    static byte[] _logoBytes;
    public static byte[] Logo
    {
	get
	{
	    // Returns logo image bytes.
	    if (_logoBytes == null)
	    {
		_logoBytes = File.ReadAllBytes("Logo.png");
	    }
	    return _logoBytes;
	}
    }
}

File.WriteAllBytes

The File.WriteAllBytes method does exactly as you might expect: it writes the bytes in a byte array to a file at the location you specify. You can see an example of this on this site in a method that compresses byte data.

TextReader/TextWriter

The TextReader and TextWriter types form the base class that other, more useful types derive from. They are not really useful on their own most of the time.

BinaryReader/BinaryWriter

The .NET Framework has two types that can make reading or writing a binary file much easier: the BinaryReader and BinaryWriter types. These abstract data types make it unnecessary for you to ever have to deal with the raw bits and bytes yourself.

CSV files

Comma-separated values files are text-based databases. With the System.IO namespace, you can read them into your C# program. The TextFieldParser type is also available, but may incur a performance loss.

File equality

How can you tell if two files are exactly equal? Unfortunately, the file system's metadata is not sufficient. For this reason, using the File.ReadAllBytes method and comparing the byte arrays is necessary.

Directory

You can manipulate directories on the file system with your C# program. The Directory static type is necessary for this. You can also test to see if a directory exists.

Manipulate files

You can copy, delete, rename, or get time information about files using the C# programming language. These actions are available through the File type as well as the FileInfo type.

Seek

You can seek to a specific location in your file with the Seek method. We demonstrate this method and provide a detailed benchmark of its best usage.

FileInfo

You can get information about a file from the file system with the FileInfo type. This does not require you to load the entire file into memory. It does access the storage unit however.

Sizes

There are other ways to get the sizes of files and directories—you can do this in different ways depending on what objects you are using to manipulate the file. You can often determine file lengths without reading in the entire file.

Stream

Streams take many forms in the .NET Framework. Sometimes, leaving a file on the disk would impact performance or stability in a negative way. In these cases, you can use a MemoryStream to store a file in memory in Stream format. This is described further.

WebClient

Not every file you want to use is always located on your local disk. Instead, you may need to access the network to download a file from a remote computer. The WebClient type provides a way for you to do this.

Office

It is common to need to control Microsoft Excel with C# code. We first introduce an approach to Excel interop that is very fast. Microsoft Word is also controllable with the interop assemblies; we show how to read in each word from a document in a loop.

MemoryMappedFile

It is possible in version 4.0 of the .NET Framework to map files into memory with the MemoryMappedFile class. We present a benchmark that reveals the performance advantage of using MemoryMappedFile.

Performance

Most every time you access a file in Windows, the operating system itself puts that file into a memory cache. We provide a benchmark of file system caches, and also dive into how temporal locality can be used to improve performance.

Summary

We saw several methods and patterns for using System.IO in the C# programming language. The .NET Framework is excellent with file handling, and one benchmark he has seen measures it as even faster than C++ on Windows in its default configuration. Nearly every medium or larger size program will need to use file input/output, and we demonstrated some of the clearest methods for this purpose.