Use LINQ to page through a collection in memory with LINQ to Objects

It is not uncommon to find myself needing to page through a collection of .NET objects in memory. LINQ to objects provides a great way to accomplish paging using the Skip and Take extension methods.

The following example code illustrates the pattern in C#.

List<int> list = new List<int> { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
int chunkSize = 3, index = 0;
IEnumerable<int> chunk = list.Take(chunkSize);

while( chunk.Count() > 0 )
{
	// Do something with the chunk of data.
	chunk.Dump(); // This shows the data in LINQPad.
	
	index += chunkSize;
	chunk = list.Skip(index).Take(chunkSize);
}

The results of the above code in LINQPad:

A common scenario that arises is the need to run a database query with an in clause across a large amount of data in memory. A where clause with a condition comparing to a list in an ORM tool normally results in an IN clause with bound parameters. Since SQL Server can have a limited number of entries in an IN clause, somewhere around 2,000, processing can easily be done by paging through the data in memory.

Happiness :-)