My first thought was to simply replace a stepper motor in the typical belt drive assembly with a BLDC motor and then add an encoder... somewhere. I never decided where to put the encoder because it wasn't long before I realized that that particular approach wouldn't work too well, at least without its fair share of software complications. One of the specs on every BLDC is the number of poles, or pole-pairs (there are always twice as many poles and there are pole-pairs). While not exactly the same, [number of poles] x 3 (or [number of pole-pairs] x 6) can be considered to be the equivalent of the number of steps per revolution in a stepper motor. The problem is that the number of poles on a BLDC is quite small in comparison to a stepper motor, so driving an axis directly wouldn't have the required positioning accuracy, at least not with a much more complicated motor control software. (The robotics industry has figured it out and use it all the time. It's roughly the equivalent of micro-stepping, I think...)
I thought through a few other design ideas all centered around some kind of speed reduction (gears, pulleys, threaded shafts) which has the advantage of increasing the effective number of steps in a revolution by the same factor that speed is reduced by. For example, an 8:1 gear reduction means that for every 8 revolutions on the input shaft (motor end), there is 1 revolution on the output shaft (connected to the drive axis). The speed is reduced by 8 while the effective [number of poles] x 3, or "steps per revolution" is multiplied by 8. Eventually, I landed on this design for one of the drive axes. It uses a 14 pole "medium speed" motor (1200kv), a 8:1 gear reduction from the motor to drive shaft (yellow and green features), and a 10 threads per inch shaft for even further speed reduction (torque multiplication). All in all this would give 80 motor revolutions per inch of travel, plenty (overkill?) of positioning accuracy without the need for any micro-stepping.
But I still had two problems. One, to get good low speed torque and accurate positioning out of a BLDC motor it really should be a sensored motor, meaning it has 3 hall effect sensors in the motor aligned so that as the magnets of the motor rotate the sensors will tell the controller when to turn on or off each of the three phases in the motor. (See the first 4 pages of this application note for a good explanation of BLDC commutation). The problem I had here is that the only sensored motors I could find were either far too few poles or way too expensive. One possible remedy to this problem, I thought, was to use the signal from the encoder to commutate the motor, thus negating the need for a sensored motor. The other problem was that I underestimated the price of quality encoders--the cheapest ones I found of acceptable quality were $40 and ENORMOUS (see that big black bulk in the image above?), smaller packaged encoders were in the $80+ range. It was almost enough to drive me back in the direction of stepper motors.
But one day as I was discussing the matter with a friend at work it dawned on me: it should be possible to design a small (cheap) circuit board that mounts to the motor and adds sensors to an otherwise sensorless BLDC motor. I did some research and found that other people have similarly hacked their motors and so I drew up a quick PCB to fit my motor. As I was going through that process I had a second epiphany: I should be able to use the sensors on the motor in a similar way as an encoder to get position feedback! It would be a much coarser encoder than I was planning on, but if I had a properly geared drive system so that I didn't need "micro-stepping" than I was never going to use that extra encoder resolution anyway.
I finished designing the PCB, which was small enough that it only cost a few dollars, and added high quality hall effect sensors. The end result was something around $6 for a sensored hack to my motor, and it worked perfectly (see here for the details of the design process). The downside to these sensors is that they will only work with a specific motor.
Another aspect that I'm happy with on this design is that being direct drive I eliminate some concerns I was having about backlash in my gear system. That is, if the gears don't perfectly mesh then whenever the direction of rotation reverses there's some "dead space" where the motor is turning (and reading position changes) but the drive shaft isn't actually doing anything. In practice all gears have some amount of backlash. There's ways of dealing with it in "zero backlash" which are actually two gears loaded under a spring to take up the backlash, but those are expensive and hard to come by. But with direct drive I don't have to deal with any of that.
Also, with this design the motor sits prominently up on top of the axis, visible for all to see. This is further accentuated by the fact that the motor is an outrunner, meaning the part that rotates is the outer most diameter of the motor, which will look pretty cool watch.