You could try weld the driver to the hub through the brake side. Might not survive it, though. And it would be super hard to grease if you needed to.
edit, nvm, you could grease it pretty easily. Still, would not be the easiest thing, considering how cheap a simple fixie hub is.