[Tux3] Basic SMP locking for balloc

[Daniel Phillips]

Allocations from multiple sources can be going on in parallel, so
balloc and bfree have to be serialized.  Allocations  come from files,
inode table, atom table, and even the allocation map itself.

What happens when an allocation creates a new block in the allocation
map?  There is danger of recursion here: with this simple locking
the bitmap->i_mutex is held by the caller of balloc, and if balloc
calls balloc it will deadlock trying to acquire bitmap->i_mutex, which
the task already holds.  What saves us is, balloc works in cache, and
allocation is only done when the bitmap is synced to disk.  The caller
at that time will be vfs, which is busy flushing the block device, and
does not take the inode->i_mutex.  Later, the caller will be our own
delta commit code as we take over responsibility of disk syncing from
the vfs to implement atomic commit.

We do have to worry about other balloc-in-balloc recursions, for
example, we have to be careful about balloc changing a bit in a block
that belongs to the delta being synced to disk.  We will deal with that
interesting recursion later. For now, I think the simple locking here
will work for now.

diff -r 975db73a0cb8 user/kernel/balloc.c
--- a/user/kernel/balloc.c	Sat Dec 27 12:14:55 2008 -0800
+++ b/user/kernel/balloc.c	Sat Dec 27 15:23:05 2008 -0800
@@ -226,14 +226,17 @@ block_t balloc(struct sb *sb, unsigned b
 {
 	assert(blocks > 0);
 	trace_off("balloc %x blocks at goal %Lx", blocks, (L)sb->nextalloc);
+	mutex_lock(&sb->bitmap->i_mutex);
 	block_t goal = sb->nextalloc, total = sb->volblocks, block;
 	if ((block = balloc_from_range(sb->bitmap, goal, total - goal, blocks)) >= 0)
 		goto found;
 	if ((block = balloc_from_range(sb->bitmap, 0, goal, blocks)) >= 0)
 		goto found;
+	mutex_unlock(&sb->bitmap->i_mutex);
 	return -1;
 found:
 	printf("balloc extent -> [%Lx/%x]\n", (L)block, blocks);
+	mutex_unlock(&sb->bitmap->i_mutex);
 	return block;
 }
 
@@ -244,6 +247,7 @@ void bfree(struct sb *sb, block_t start,
 	unsigned mapmask = (1 << mapshift) - 1;
 	unsigned mapblock = start >> mapshift;
 	char *why = "could not read bitmap buffer";
+	mutex_lock(&sb->bitmap->i_mutex);
 	struct buffer_head *buffer = blockread(mapping(sb->bitmap), mapblock);
 	printf("free <- [%Lx]\n", (L)start);
 	if (!buffer)
@@ -254,10 +258,12 @@ void bfree(struct sb *sb, block_t start,
 	brelse_dirty(buffer);
 	sb->freeblocks += blocks;
 	//set_sb_dirty(sb);
+	mutex_unlock(&sb->bitmap->i_mutex);
 	return;
 eeek:
 	why = "blocks already free";
 	brelse(buffer);
 eek:
 	warn("extent 0x%Lx %s!\n", (L)start, why);
+	mutex_unlock(&sb->bitmap->i_mutex);
 }

_______________________________________________
Tux3 mailing list
Tux3 at tux3.org
http://mailman.tux3.org/cgi-bin/mailman/listinfo/tux3