Poster: Cladhaire at 7/20/2006 11:43:34 AM PDT
Subject: Slouken: Issue with bit.bnot()? |
| |
Rophy was doing some testing today and he ran into the following situation:
bit.band(3, bit.bnot(0) is returning 0, instead of the expected result (3), specifically the following:
ChatFrame1:AddMessage(bit.band(3, bit.bnot(0))
Anything wonky going on that you can see, when you get a moment?
http://cladhaire.wowinterface.com
|
| | http://forums.worldofwarcraft.com/thread.aspx?fn=wow-interface-customization&t=418230&p=#post418230 |
| |
Poster: slouken at 7/20/2006 3:15:34 PM PDT
Subject: Re: Slouken: Issue with bit.bnot()? |
| |
Q u o t e:
Does the maximum value of an unsigned long long fit without precision loss inside of a double?
No, it doesn't. I think that's the problem here. I'm not sure how it works in the unmodified bitlib... ?
|
| | http://forums.worldofwarcraft.com/thread.aspx?fn=wow-interface-customization&t=418230&p=#post418393 |
Poster: slouken at 7/20/2006 6:57:14 PM PDT
Subject: Re: Slouken: Issue with bit.bnot()? |
| |
Okay, switched bitlib back to 32 bits for 1.12.
bnot(0) == 4294967295
bnot(4294967295) == 0
|
| | http://forums.worldofwarcraft.com/thread.aspx?fn=wow-interface-customization&t=418230&p=#post418507 |
Poster: slouken at 7/20/2006 8:47:26 PM PDT
Subject: Re: Slouken: Issue with bit.bnot()? |
| |
Q u o t e:
I seem to recall the switch to 64 bit was made to address a problem with the other 32 bits of the double not being properly masked to 0 under some conditions (right shifts, perhaps?)
Possibly, although the Lua number type not being able to hold the full range of the bit values strikes me as a bigger problem. :)
|
| | http://forums.worldofwarcraft.com/thread.aspx?fn=wow-interface-customization&t=418230&p=#post418539 |
Poster: slouken at 7/21/2006 10:11:57 AM PDT
Subject: Re: Slouken: Issue with bit.bnot()? |
| |
*laugh* That t0ta11y r0xx0rzzz!21!
|
| | http://forums.worldofwarcraft.com/thread.aspx?fn=wow-interface-customization&t=418230&p=#post418806 |
Poster: slouken at 7/25/2006 3:47:23 PM PDT
Subject: Re: Slouken: Issue with bit.bnot()? |
| |
Q u o t e:
I'm hoping that the results make it back into lua as:
unsigned int -> double
Rather than
int -> double
I'd hope that'd take care of it.
Yep, and it does.
|
| | http://forums.worldofwarcraft.com/thread.aspx?fn=wow-interface-customization&t=418230&p=#post421090 |
