Deleting lots and lots and lots of data from a Sql Server table

We just had an interesting discussion among our team regarding the best way to remove a very large number of rows from a database without having the transaction log grow out of control.  The scenario is something like this:

  1. Need to delete 50 billion rows from a table
  2. Need to do it when there is only 4 G of hard drive space on the server
  3. Cannot turn off logging for the entire database

At this point the only option that we've come up with is creating a stored procedure that deletes a range of records and then shrinks the database, as well as the transaction log.  The execution of the stored procedure would delete approximately 250,000 records at a time. 

Unfortuntely when posed with the question of how to best accomplish this feat this is all I (as well as others) could come up with.  Is there a better way?

# re: Deleting lots and lots and lots of data from a Sql Server table

Friday, September 19, 2003 3:37 AM by Paul    
Well you could use truncate on the table, which doesn't write to the translog for every record, but doesn't support a where clause (So it's kinda all or nothing solution).

# re: Deleting lots and lots and lots of data from a Sql Server table

Friday, September 19, 2003 5:02 AM by Addy Santo    
How many rows do you plan to leave in the table? It might be more efficient to do a Bulk Copy of the existing records to a seperate table,run a Truncate on the table, and the Bulk Copy back. None of those activities are logged to the translog. That would obviously work best if you are deleting the large majority of the records.

# re: Deleting lots and lots and lots of data from a Sql Server table

Tuesday, September 23, 2003 8:02 PM by Peter    
You could improve the stored procedure a bit by incrementing the number of rows you're going to delete by each calling of the procedure. First time you delete 250,000 next time n + 250,000 rows. Well, the math theory says that "n" alghoritm and "n + m" is still O(n), so the performance well be just slightly better. Sorry, I'm not a db guru and this couldn't be very useful.

Post a Comment

 
 
Prove you're not a spammer: 
2 + 2 =