IMHO it would be better to make adjustment of user color by sliders.

It is true that it would be easier ..
Some tips that may help:

- a right click on a default color can prefill the values of the clicked color
- when a textbox has focus, the mouse wheel can change the value (press CTRL while rolling the wheel for a faster value change)