Data Problem.

[tg]

so, i've got a problem....

heres the pertinent background info ---
i've got a table that will hold data for graphs.
the actual data field will contain both percentages and whole numbers.
the data type for the data field is 'float'
the data is moved back and forth between flash and sql server (v7) via xml.


heres the problem ---
when a user types into the form 43.63 (a percentage) it gets loaded into the db, and i can see in the db table that it is in fact 43.63

but when it gets reloaded into flash i get: 43.630000000000003

or sometime the user will put in 61.44 and when it is reloaded into flash i get back: 61.439999999999993.

i know i can fix the by rounding off to 2 significant digits, but i was wondering if any of you db gurus know of a different solution.

thanks.

[CyanBlue]

so, i've got a problem....

Welcome to the AA, tg... :D

Like I know anything about the SQL, but there's got to be some SQL syntax which you can use to limit the number of precision like you can see in this table...
http://www.techonthenet.com/sql/datatypes.htm

[freddycodes]

But you say in the database its stored correctly? When you create the xml, if you view the xml its in the longer format?

What code are you using to produce the xml?

[tg]

this is the code that builds the xml from the form:

buildData=function(){
var xObj = new XML();
xObj.appendChild(xObj.createElement("DataPoint"));
xObj.firstChild.attributes.val = subForm.dataVal_txt.text;
if(updateType=='edit')xObj.firstChild.attributes.o ldID=formatDate(subForm.arOldDate[0],(subForm.arOldDate[1]-1));
xObj.firstChild.attributes.id = formatDate(subForm.year_cb.getSelectedItem().label ,setDateVals());
xObj.firstChild.attributes.msr = CurrentObject._id;
//trace(xObj);
return xObj;
};

this is the (cf) code that extracts the data from the db:

<cfinclude template="../conn.cfm">
<cfsetting enablecfoutputonly="yes">
<cftry>
<cfparam name="nbr" default=0>
<cfoutput><?xml version="1.0" encoding="iso-8859-1" ?></cfoutput>
<cfquery name="obj" datasource="#con_DSN#">SELECT * FROM measureData WHERE msrID=#nbr# order by dataDate</cfquery>
<cfoutput><dataPoints count="#obj.recordCount#"></cfoutput>
<cfoutput query="obj"><DataPoint msr="#msrID#" id="#DateFormat(dataDate,'yyyy,mm,dd')#" val="#dataVal#" /></cfoutput>
<cfoutput></dataPoints></cfoutput>
<cfcatch>
<cfoutput><errors page="dataForMeasures.XML"><error details="#cfcatch.Detail#" message="#cfcatch.Message#" type="#cfcatch.Type#" /></errors></cfoutput>
</cfcatch>
</cftry>

[tg]

hmm... after looking at this... maybe i should explicitly convert the data in the textfield from a string to a number with Number().... very sloppy, but i'm pretty sure that that is not the problem, since when i open the table in sqlserver, it shows there correctly... unless its (sqlserver) has some built in formatting function in table view (like access does), and not showing the data in its true form.

[tg]

Welcome to the AA, tg...
whats AA?

[freddycodes]

Okay so if you just call the cfc directly, not sure you can not too familiar with CF. What do the numbers look like in the resulting xml?

Or even better if you trace the value of the xml object in flash when it receives it, what format are the numbers in?

[tg]

yeah, they're comming out of the db hosed.... of the cf is doing something to it:

<?xml version="1.0" encoding="iso-8859-1" ?>
<dataPoints count="1">
<DataPoint msr="51" id="2004,06,01" val="8.4399999999999995" />
</dataPoints>

[freddycodes]

So I think you want to format it in your output from cf
http://livedocs.macromedia.com/coldfusion/6/CFML_Reference/functions-pt247.htm

[tg]

freddycodes.
thankyou, you've done it again. altho i cant use NumberFormat (because i'm still stuck with cf4.5) i was able to use 'DecimalFormat'.

again. thanks a bunch.

[CyanBlue]

AA is the club for the ActionScript Addicts... :D