`Result::unwrap()` on an `Err` value: Conversion("cannot interpret F64(Some(0.0)) as an f32 value")

[TheRustifyer]

Hello. I am using tiberius on some database ORM, and I am unable to play with float types.

Whenever I define a classical float on SqlServer, tiberius always broke the result on the row.get() method.

Is this a know bug? Whatching the crate tiberius-derive, they specifically talk about this one.

Known issues:

There is a tiberius bug(?) that breaks Option fields at the moment. Tiberius appears to always parse null Float(53) (double) columns as Nullable(f32) independently of the column type.

So, my Rust field is an Option. If I try to change it to an Option...

Result::unwrap()` on an `Err` value: Conversion("cannot interpret F32(None) as an f64 value")

Is this a known bug? Exists some workaround over it? How can I fix my code? I am using it on an enterprise level application, releasing the code in production, and we're getting delayed because we can't work with our float columns.

Thanks in advice for your time.

[TheRustifyer]

I downloaded the tiberius source-code. Error is on the fron_sql()! macro for floats.

I manually implemented FromSql<'a> for f32 and f64.
Having a field determined as an Option<i64>, I get these results:

Data as F64: Some(11.3)
Data as F64: Some(11.3)
Data as F64: Some(11.5)
Data as F64: Some(15.5)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(10.0)
Data as F64: Some(0.0)
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as F64: Some(0.0)
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as F64: Some(0.0)
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as F64: Some(0.0)
Data as f32: None
Data as F64: Some(0.0)
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as F64: Some(0.0)
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as F64: Some(0.0)
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as F64: Some(0.0)
Data as F64: Some(0.0)
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None
Data as f32: None

On an Option Rust variable, tiberius is retrieving from an SqlServer column defined as float - null, values when
present as f64, and null values as optional of f32.

I literally just matched against bot ColumnData of F32 and F64 variants. Error should be more deep that the FromSql impl from floats. I have not enought time to investigate it, but I hope that this leads to the tiberius dev team to find such bug.

Meanwhile, I will open a PR for solving the issues. Code it's provisional, as it shouldn't be a full correction, but at least
tiberius now is able to play with Option and Option values.

I would like to work with you when I have some time. I also would like that the dev team take my PR and performn the full correction over it, so I can contribute to the tiberius ecosystem, and be aware of the solution implemented by you.

[pimeys]

Hey, we have a holiday today so probably not answering super fast. Send a PR if you think you found a bug!

At least here we do detect the float size even for nulls. And here we provide conversion functions for both: f32 and f64, including nulls.

[TheRustifyer]

Hello @pimeys thanks for your response even on holidays. I am forking the project now to make a pull request. Solution provided is not beatiful, but stops the breakage when a float value is required. I will make the PR and the dev-team can take a look at it.

Here is exactly the problem. Float null values are getting parsed as ColumnData::F32, and any non null value it's being retrieved as ColumnData::F64. That's such an extrange behaviour, and I guess that is a deep bug (I mean, when bytes are read on the query response).

I am making the PR now

[pimeys]

If the float is nullable, it should come back as variable length column. The column type info tells us the size and we act accordingly.

I remember similar issues being fixed in this pull request: #209

Are you running the latest version?

[TheRustifyer]

Yes, I upgraded tiberius to 0.11.2. I guess that is easy to check the bug with some SqlServer instance, and some column defined as float, obv, as nullable. I can provide logs or whatever if you're on holidays, and make the debug work for you

[pimeys]

Just a test would be great!

[TheRustifyer]

I am working with (Canyon-SQL)(https://github.com/zerodaycode/Canyon-SQL), that acts as an ORM. Firstly was integrated with tokio_postgres, now with tiberius.

A simple struct to use with Canyon-SQL, with a datasource pointing to an SqlServer instance, which will trigger the tiberius connector.

#[derive(Debug, Clone, CanyonCrud, CanyonMapper)]
#[canyon_macros::canyon_entity(table_name = "Ticket", schema = "tic")]
pub struct Ticket {
    #[primary_key]
    pub TicketCode: i64,
    pub DedicatedHours: Option<f64>,
}

Canyon uses as dependency: tiberius = { version = "0.11.2", features = ["tds73", "chrono"] }

Now, a query is generated, gets the results back as a Vec<tiberius::Row>, and the CanyonMapper macro just inits the struct fields and uses:

quote! {  
    #ident: row.get::<#ty, &str>(#ident_name)
        .expect(format!("Failed to retrieve the `{}` field", #ident_name).as_ref())
 }

there's a lot more of usecases for diff types, just trying to set up a decent explanation.

After that, when a field contains an Option<f64>:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Conversion("cannot interpret F32(None) as an f64 value")'

and if I change it to an Option<f32>:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Conversion("cannot interpret F64(Some(11.3)) as an f32 value")'

After manually implementing FromSql<'a> for both f32andf64, and as commented above, tiberius is matching the
ColumnData::F32 on None values, and ColumnData::F64 on any non null value.

impl<'a> FromSql<'a> for f64 {
    fn from_sql(value: &'a ColumnData<'static>) -> crate::Result<Option<Self>> {
        match value {
            ColumnData::F32(data) => {  // Here are always the None values
                Ok(data.map( |v| v as f64))
            },
            ColumnData::F64(data) => {  // Here are always the non null-values
                Ok(*data)
            },
            v => {
                Err(crate::Error::Conversion(
                    format!("cannot interpret {:?} as an f64 value", v).into(),
                ))
            }
        }
    }
}

I guess that this fix is not solving the issue.

[pimeys]

What is the exact table definition in the database? Nullable reals or nullable floats?

[TheRustifyer]

nullable float

[pimeys]

I wrote a few tests, but I can't really reproduce this:

#[test_on_runtimes]
async fn nullable_f64<S>(mut conn: tiberius::Client<S>) -> Result<()>
where
    S: AsyncRead + AsyncWrite + Unpin + Send,
{
    let table = random_table().await;

    conn.execute(format!("CREATE TABLE ##{table} (m1 float NULL)"), &[])
        .await?;

    conn.execute(&format!("INSERT INTO ##{table} (m1) VALUES (null)"), &[])
        .await?;

    let row = conn
        .query(&format!("SELECT m1 FROM ##{table}"), &[])
        .await?
        .into_row()
        .await?
        .unwrap();

    let col: Option<f64> = row.get(0);
    assert_eq!(None, col);

    Ok(())
}

#[test_on_runtimes]
async fn nullable_f32<S>(mut conn: tiberius::Client<S>) -> Result<()>
where
    S: AsyncRead + AsyncWrite + Unpin + Send,
{
    let table = random_table().await;

    conn.execute(format!("CREATE TABLE ##{table} (m1 real NULL)"), &[])
        .await?;

    conn.execute(&format!("INSERT INTO ##{table} (m1) VALUES (null)"), &[])
        .await?;

    let row = conn
        .query(&format!("SELECT m1 FROM ##{table}"), &[])
        .await?
        .into_row()
        .await?
        .unwrap();

    let col: Option<f32> = row.get(0);
    assert_eq!(None, col);

    Ok(())
}

Both of these are green with the latest main branch.

[TheRustifyer]

can you point me where do you wrote those tests? So I can download them to my environment and check them out?

[pimeys]

#251

Have one of the SQL Servers running defined in the docker compose setup, run cargo test nullable_f.

[TheRustifyer]

Sorry, I am unable to reach the server from my location.

Error: Io { kind: ConnectionRefused, message: "... (os error 10061)" }

Updated from the upstream and the actions are launching now, but I must assume that it will be exactly the same results... even tho, I merged my changes into my main branch, and something can break.

What more info can I provide you to demonstrate the error I am fetching when run my code with the latest relese of tiberius? 0.11.2

[pimeys]

Maybe try to modify the tests if they're doing something different than you do. The CI runs all the tests, so you can take a look into .github directory and see how we initialize the test env.

CI for that PR I created is green.