Shortly after the last release of Database Schema Reader, here's another release.

Last time I added a CopyToSQLite tool, which reads (almost) any database and creates a SQLite clone. I also added experimental support for SQLServer CE 4.0, but it had some big limitations.

So the obvious next step was to fix some of those limitations, and that's what this release focuses on. Unfortunately CopyToSQLite.exe is now a little misnamed.

I added a little bit of conversion code so VARCHAR(MAX) found in the origin database gets converted to NTEXT (and VARBINARY to IMAGE). The reading of SQLServer CE 4 databases got improved too (capturing the  foreign and unique keys and identity columns).

SQLServer allows an integer column to be marked as "Identity" so it will be auto-generated (the equivalent in Oracle is to create a sequence and an insert trigger). But you can't include the identity column when you insert a row. So, when cloning data, SQLServer also allows identity-inserts with SET IDENTITY_INSERT [MyTable] ON/OFF. When you've done that, you should reset the identity seed with DBCC CHECKIDENT.

But in SQLServer CE 4, there is no DBCC CHECKIDENT. You have to do an ALTER TABLE [MyTable] ALTER COLUMN [IdentityColumn] IDENTITY (999,1) (where 999 is the new max(IdentityColumn)). It's another of those little gotchas between SQLServer and SQLServer CE.

SQLServer CE 4 has some great new paging syntax:

SELECT Id, Name FROM MyTable
ORDER BY Name
OFFSET 10 ROWS
FETCH NEXT 5 ROWS ONLY

It's similar to the LIMIT/OFFSET syntax in MySQL and SQLite, but (reasonably enough) must follow an ORDER BY. Rather than offset/skipping a number of rows, I'd like to specify just page number and page size - or another formula. The SQLServer 2011/Denali documentation shows the offset/fetch syntax for a start row/end row expression:

SELECT DepartmentID, Name, GroupName
FROM HumanResources.Department
ORDER BY DepartmentID ASC
OFFSET @StartingRowNumber - 1 ROWS
FETCH NEXT @EndingRowNumber - @StartingRowNumber + 1 ROWS ONLY

But that doesn't work in SQLServer CE 4. You can use parameters, or constants, or expressions with constants, but apparently not expressions with constants and parameters. Oh well, it's still much better than horrible OVER subqueries.

Overall, the "CopyToSQLite" to SQLServer CE 4 seems to work well. Using easily deployable file databases like SQLServer CE and SQLite is simple and powerful.

My Codeplex project, Database Schema Reader, has a new version.

I needed to create a SQLite database was a replica of a parent SQL Server database. The existing project could easily read the schema, and the code generation tools could give me the table DDL and the insert SQL. It was just a matter of executing the SQL and creating the database file.

So I created another simple Windows Forms UI. The SqlWriter class needed some tweaks for SQLite support (and I added integration tests).

For fun, I decided to see if it could also support Microsoft's latest version of SQL Server CE 4.0.

SQL Server CE 4.0 is, like SQLite, an in-process database which can be XCOPY deployed. And unlike previous versions, it is easily able to run ASP.Net websites (it's the default database behind WebMatrix). And 4.0 provides the standard ADO GetSchema which my Database Schema Reader uses (3.5 throws a Not Implemented exception).

It has some of the same limitations as SQLite, such as no stored procedures (good riddance), and no output parameters. But it also has additional limitations. In SQLite you can batch together multiple SQL statements in a single line and execute them in one command. You can't in SQL Server CE.

Unlike SQLite's weakly typed and very simple data types, SQL Server CE 4.0 has most of the standard data types as SQL Server Express and full versions. Unfortunately, this was the big problem. If your table is defined with an IDENTITY primary key, you can't insert the data row with the same primary key, and subsequent foreign key relationships are broken. It's a pretty critical limitation for my scenario.

SQL Server CE 4 only supports a subset of SQL Server data types. The one data type it didn't support, which my database (and later versions of Northwind) use, is NVARCHAR(MAX). You have to use the horrible old NTEXT data type. I could have written a DDL provider that translates varchar max to ntext, but I don't think it's worth the effort.

So my SQL Server CE creation program has major limitations, compared to the SQLite version. It's built into CopyToSQLite.exe, and auto-detects if you've installed SQL Server CE, but you have to have a very simple database for it to work.

SQL Server CE 4 also doesn't support the ADO Sync framework which allows you to synchronize a disconnected database to a parent SQL Server database. Even SQL Server 2008 R2 Management Studio doesn't support it (you have to use Visual Studio 2010 with SP1).

The new FETCH /OFFSET syntax for paging, that will be SQL Server 11, is very nice though. I'm looking forward to that.

A long time ago I wrote my database schema reader. It was 2005, .Net 2.0 was just out and I was learning about the new features. .Net 2.0's ADO DbProviderFactory had a nice idea, GetSchema, to get database independent schema information. But the GetSchema collections were slightly different for each database, and they were data tables. So I wrote a simple facade over them to get proper classes.

Initially I used it to generate CRUD stored procedures, and then to generate data access code, even including NHibernate mappings. I even did some (simple) database conversions. All this was quite icky, and each time I did the code-gen a different way, but the core facade worked well.

After getting a couple of queries about the extracts I'd already posted, I thought I might as well put up the entire source code. So here's the codeplex project with the source code, and even some basic SQL code-gen.

I picked Codeplex just because it's more .Net-centric than Google code or github, plus I'm more familiar with TFS and Subversion source control. It was pretty easy. I don't expect many downloads (if any!), but maybe a couple of people can steal the relevant bits of source code.

Check out the database schema reader codeplex project.