-
-
Notifications
You must be signed in to change notification settings - Fork 18.7k
Closed
Labels
BugGroupbyNuisance ColumnsIdentifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.applyIdentifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.applyNumeric OperationsArithmetic, Comparison, and Logical operationsArithmetic, Comparison, and Logical operations
Description
Perhaps the following groupby aggregation should work only the numeric columns, as they would when using the dataframe:
In [1]: df = pd.DataFrame({'bar': {0: 1, 1: 1, 2: 1}, 'foo': {0: 0, 1: 1, 2: 2}, 'foo1': {0: 1, 1: 2, 2: 3}, 'hello': {0: 'a', 1: 'a', 2: 'a'}}, columns=['bar', 'foo', 'foo', 'hello'])
In [2]: df
Out[2]:
bar foo foo hello
0 1 0 1 a
1 1 1 2 a
2 1 2 3 a
In [3]: df.mean() # hello is ignored
Out[13]:
bar 1
foo 1
foo 2
dtype: float64
In [4]: df.groupby(level=0, axis=1).mean()
---------------------------------------------------------------------------
DataError Traceback (most recent call last)
<ipython-input-4-7c2612a8fbda> in <module>()
----> 1 df.groupby(level=0, axis=1).mean()
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/groupby.pyc in mean(self)
351 """
352 try:
--> 353 return self._cython_agg_general('mean')
354 except GroupByError:
355 raise
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/groupby.pyc in _cython_agg_general(self, how, numeric_only)
1569
1570 def _cython_agg_general(self, how, numeric_only=True):
-> 1571 new_blocks = self._cython_agg_blocks(how, numeric_only=numeric_only)
1572 return self._wrap_agged_blocks(new_blocks)
1573
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/groupby.pyc in _cython_agg_blocks(self, how, numeric_only)
1616
1617 if len(new_blocks) == 0:
-> 1618 raise DataError('No numeric types to aggregate')
1619
1620 return new_blocks
DataError: No numeric types to aggregate
From this SO question, where I gave very hacky workaround.
cc #3683 @jreback was this the question you were talking about? This ones related but in the sense of coming up against non unique problems... Thought I should mention it here anyway.
Metadata
Metadata
Assignees
Labels
BugGroupbyNuisance ColumnsIdentifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.applyIdentifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.applyNumeric OperationsArithmetic, Comparison, and Logical operationsArithmetic, Comparison, and Logical operations
Type
Projects
Milestone
Relationships
Development
Select code repository
Activity
jreback commentedon May 23, 2013
this is the question, but still breaks (even on my new branch).... will look at this soon
jreback commentedon May 23, 2013
linking to #3679
keir commentedon Apr 29, 2014
Any update on this @jreback? Did your branch ever go anywhere?
jreback commentedon Apr 29, 2014
you are welcome to do a pr if u would like
WillAyd commentedon Jul 6, 2018
Still a problem, though note that this only fails due to
axis=1
[-]Groupby aggregations could ignore non-numeric columns[/-][+]Groupby aggregations could ignore non-numeric columns when axis=1[/+]NumberPiOso commentedon Feb 13, 2022
Nowadays running the example results in a FutureWarning at both operations.
This change was introduced by PR #41480. So, nowadays groupby aggregations should NOT ignore non-numeric columns. Closing this issue.