Moving a selection

[craigdsp]

i'm trying to write a little command that will move my selection to whole pixels.

i though originally that i would be able to write it so that i can select a bunch of things and have jsfl loop through them and move each one onto whole pixels. this failed miserablly so i'm now attempting to just get the selection moved onto a whole pixel which works with a 0.1% margin of error, which obviously dosen't work. grr it's winding me up.


dom = fl.getDocumentDOM();
rect = dom.getSelectionRect();
l = (Math.ceil(rect.left) - rect.left);
t = (Math.ceil(rect.top) - rect.top);
alert(l)
alert(t)
dom.moveSelectionBy({x:l, y:t});

[craigdsp]

ok so if i switch from using Math.ceil() to using Math.floor() it works , observe:


dom = fl.getDocumentDOM();
rect = dom.getSelectionRect();
l = (Math.floor(rect.left) - rect.left);
t = (Math.floor(rect.top) - rect.top);
dom.moveSelectionBy({x:l, y:t});


however this isn't truley moving it to the nearest pixel, it's moving it to the lowest whole pixel. not good. how can i make it behave like Math.round() without the errors??

[craigdsp]

ok so this is working like 99% :

dom = fl.getDocumentDOM();
rect = dom.getSelectionRect();
alert(rect.left);
alert(Math.round(rect.left))
alert(rect.top);
alert(Math.round(rect.top))
if (rect.left < Math.round(rect.left)) {
l = (Math.floor(rect.left) - rect.left)+1;
} else {
l = (Math.floor(rect.left) - rect.left);
}
if (rect.top < Math.round(rect.top)) {
t = (Math.floor(rect.top) - rect.top)+1;
} else {
t = (Math.floor(rect.top) - rect.top);
}
dom.moveSelectionBy({x:l, y:t});

sometimes however it will land on either xxx.1 or xxx.9 positions, this tends to occur when the original begins xxx.049xxxxxxx obviously where i am adding 1 to counter the Math.floor() when needed i am in fact adding 1.049 am i right in thinking that?? is it just that the Math class can't handle these numbers or what???

[hangalot]

i think this is a floting point math issue. what you can try is to execute two math.round's on it... a hack but it might work