Skip to content

BUG/ENH: dtype change on enlargement #6485

Open
@jreback

Description

@jreback
Contributor

http://stackoverflow.com/questions/22044766/adding-row-to-pandas-dataframe-changes-dtype/22045011#22045011

A bit non-trival (maybe coerce back after) if possible?

maybe just change this to an actual append?

In [1]: df = DataFrame({'a' : range(10)}, dtype='i4')

In [2]: df
Out[2]: 
   a
0  0
1  1
2  2
3  3
4  4
5  5
6  6
7  7
8  8
9  9

[10 rows x 1 columns]

In [3]: df.dtypes
Out[3]: 
a    int32
dtype: object

In [4]: df.loc[10] = 99

In [5]: df
Out[5]: 
     a
0    0
1    1
2    2
3    3
4    4
5    5
6    6
7    7
8    8
9    9
10  99

[11 rows x 1 columns]

In [6]: df.dtypes
Out[6]: 
a    float64
dtype: object

Activity

added this to the 0.15.0 milestone on Feb 26, 2014
modified the milestones: 0.17.0, 0.16.0 on Feb 19, 2015
goretkin

goretkin commented on Sep 30, 2016

@goretkin

Is there a way to just assert the type on an enlargement?

I'm trying to avoid this...

In [1]: import pandas as pd

In [2]: names_df = pd.DataFrame(
   ...:             {"id":pd.Series([], dtype='int64')
   ...:              })
   ...: names_df.dtypes
   ...: 
Out[2]: 
id    int64
dtype: object

In [3]: names_df.loc[0, "id"] = 3
   ...: names_df.dtypes
   ...: 
Out[3]: 
id    float64
dtype: object
jorisvandenbossche

jorisvandenbossche commented on Sep 30, 2016

@jorisvandenbossche
Member

You can avoid this by not assigning with enlargement.
Options are populating the dataframe on construction, or using concat

mroeschke

mroeschke commented on Jul 6, 2018

@mroeschke
Member

Adding to a non-empty dataframe no longer enlarges (Jeff's example) but the empty case still does.

mroeschke

mroeschke commented on Oct 22, 2019

@mroeschke
Member

The first example could use a regression test since it works on master.

In [49]: In [1]: df = DataFrame({'a' : range(10)}, dtype='i4')
    ...:

In [50]: df
Out[50]:
   a
0  0
1  1
2  2
3  3
4  4
5  5
6  6
7  7
8  8
9  9

In [51]: df.dtypes
Out[51]:
a    int32
dtype: object

In [52]: In [4]: df.loc[10] = 99
    ...:

In [53]: df.dtypes
Out[53]:
a    int64
dtype: object

In [56]: pd.__version__
Out[56]: '0.26.0.dev0+627.gef77b5700'
added and removed
Dtype ConversionsUnexpected or buggy dtype conversions
IndexingRelated to indexing on series/frames, not to indexes themselves
on Oct 22, 2019

10 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugDtype ConversionsUnexpected or buggy dtype conversionsIndexingRelated to indexing on series/frames, not to indexes themselvessetitem-with-expansion

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @goretkin@jreback@jorisvandenbossche@jbrockmendel@mroeschke

        Issue actions

          BUG/ENH: dtype change on enlargement · Issue #6485 · pandas-dev/pandas