U /eT-@sddlZddlZddlZddlmZmZddlm Z ddl m Z ddl m Z mZmZmZddlmZddlmZe rxenejZddd dd d efd d Zdd dZdddZdS)N) is_list_like is_scalar)methods) PANDAS_GT_200) DataFrameSeriesapply_concat_applymap_partitionshas_known_categories)M_Fc s$ttjtjfr4tjf|||||||d|Sd} d} ttrjtsXt| tst| nxttr|dkrj dk rt| j j dgdj }ntfdd |Dst| tfd d |Dst| j jjd d } tj| j} t| f|||||||d|S) a Convert categorical variable into dummy/indicator variables. Data must have category dtype to infer result's ``columns``. Parameters ---------- data : Series, or DataFrame For Series, the dtype must be categorical. For DataFrame, at least one column must be categorical. prefix : string, list of strings, or dict of strings, default None String to append DataFrame column names. Pass a list with length equal to the number of columns when calling get_dummies on a DataFrame. Alternatively, `prefix` can be a dictionary mapping column names to prefixes. prefix_sep : string, default '_' If appending prefix, separator/delimiter to use. Or pass a list or dictionary as with `prefix.` dummy_na : bool, default False Add a column to indicate NaNs, if False NaNs are ignored. columns : list-like, default None Column names in the DataFrame to be encoded. If `columns` is None then all the columns with `category` dtype will be converted. sparse : bool, default False Whether the dummy columns should be sparse or not. Returns SparseDataFrame if `data` is a Series or if all columns are included. Otherwise returns a DataFrame with some SparseBlocks. .. versionadded:: 0.18.2 drop_first : bool, default False Whether to get k-1 dummies out of k categorical levels by removing the first level. dtype : dtype, default bool Data type for new columns. Only a single dtype is allowed. .. versionadded:: 0.18.2 Returns ------- dummies : DataFrame Examples -------- Dask's version only works with Categorical data, as this is the only way to know the output shape without computing all the data. >>> import pandas as pd >>> import dask.dataframe as dd >>> s = dd.from_pandas(pd.Series(list('abca')), npartitions=2) >>> dd.get_dummies(s) Traceback (most recent call last): ... NotImplementedError: `get_dummies` with non-categorical dtypes is not supported... With categorical data: >>> s = dd.from_pandas(pd.Series(list('abca'), dtype='category'), npartitions=2) >>> dd.get_dummies(s) # doctest: +NORMALIZE_WHITESPACE Dask DataFrame Structure: a b c npartitions=2 0 uint8 uint8 uint8 2 ... ... ... 3 ... ... ... Dask Name: get_dummies, 2 graph layers >>> dd.get_dummies(s).compute() # doctest: +ELLIPSIS a b c 0 1 0 0 1 0 1 0 2 0 0 1 3 1 0 0 See Also -------- pandas.get_dummies )prefix prefix_sepdummy_nacolumnssparse drop_firstdtypez`get_dummies` with non-categorical dtypes is not supported. Please use `df.categorize()` beforehand to convert to categorical dtype.z`get_dummies` with unknown categories is not supported. Please use `column.cat.as_known()` or `df.categorize()` beforehand to ensure known categoriesNobjectcategory)includec3s|]}t|VqdSN)ris_categorical_dtype.0cdata:/tmp/pip-unpacked-wheel-dbjnr7gq/dask/dataframe/reshape.py szget_dummies..c3s|]}t|VqdSrr rrrr r!s.r) isinstancepdrr get_dummiesrrNotImplementedErrorr dtypesany_metaZ select_dtypesrall __class__ __module__splitsysmodulesr ) rrrrrrrrkwargsZ not_cat_msgZunknown_cat_msg package_nameZdummiesrrr r%s`Z        r%meanc Csjt|r|dkrtdt|r(|dkr0tdt||sFtdt||sZtdt|rttdd|Dst|stdd d d d d g}t|r||krtdddd|Dtj ||j j |d}t|r|}ntj j t||fd|gd}|dkrzt|r6tj|||jt|j|d}nBtj|t|j|d}|D]"} || ||j| || <qTntj|tjt|j|d}|||d} |dkrt|gtjtj|d| d} |dkrt|gtjtj|d| d} |d kr| S|d kr| S|d kr| | S|d kr>t|gtjtj|d| dS|d krbt|gtjtj|d| dStdS)a Create a spreadsheet-style pivot table as a DataFrame. Target ``columns`` must have category dtype to infer result's ``columns``. ``index``, ``columns``, and ``aggfunc`` must be all scalar. ``values`` can be scalar or list-like. Parameters ---------- df : DataFrame index : scalar column to be index columns : scalar column to be columns values : scalar or list(scalar) column(s) to aggregate aggfunc : {'mean', 'sum', 'count', 'first', 'last'}, default 'mean' Returns ------- table : DataFrame See Also -------- pandas.DataFrame.pivot_table Nz.'index' must be the name of an existing columnz0'columns' must be the name of an existing columnz 'columns' must be category dtypezs'columns' must have known categories. Please use `df[columns].cat.as_known()` beforehand to ensure known categoriescSsg|] }t|qSr)r)rvrrr szpivot_table..z4'values' must refer to an existing column or columnsr2sumcountfirstlastzaggfunc must be either z, css|]}d|dVqdS)'Nr)rxrrr r!szpivot_table..)name)names)r7r8)rrindex)rr=)r=rvalues)r5r2Zpivot_table_sum)chunkZ aggregatemetatokenZ chunk_kwargs)r6r2Zpivot_table_countZpivot_table_firstZpivot_table_last)r ValueErrorrrr rr*joinr$ZCategoricalIndexcat categoriesZ MultiIndexZ from_productsortedrrZIndexr)Zastyper'npZfloat64rZ pivot_sumZ pivot_aggZ pivot_countZ pivot_firstZpivot_agg_firstZ pivot_lastZpivot_agg_last) Zdfr=rr>ZaggfuncZavailable_aggfuncsZcolumns_contentsZ new_columnsr@Z value_colr0Zpv_sumZpv_countrrr pivot_tables    "         rHvaluec Cs(ddlm}|jtj||||||ddS)a Unpivots a DataFrame from wide format to long format, optionally leaving identifier variables set. This function is useful to massage a DataFrame into a format where one or more columns are identifier variables (``id_vars``), while all other columns, considered measured variables (``value_vars``), are "unpivoted" to the row axis, leaving just two non-identifier columns, 'variable' and 'value'. Parameters ---------- frame : DataFrame id_vars : tuple, list, or ndarray, optional Column(s) to use as identifier variables. value_vars : tuple, list, or ndarray, optional Column(s) to unpivot. If not specified, uses all columns that are not set as `id_vars`. var_name : scalar Name to use for the 'variable' column. If None it uses ``frame.columns.name`` or 'variable'. value_name : scalar, default 'value' Name to use for the 'value' column. col_level : int or string, optional If columns are a MultiIndex then use this level to melt. Returns ------- DataFrame Unpivoted DataFrame. See Also -------- pandas.DataFrame.melt r) no_defaultmelt)r@id_vars value_varsvar_name value_name col_levelrA)dask.dataframe.corerJr r rK)framerLrMrNrOrPrJrrr rK<s) rK)NNNr2)NNNrIN)r.ZnumpyrGZpandasr$Zpandas.api.typesrrZdask.dataframerZdask.dataframe._compatrrQrrrr Zdask.dataframe.utilsr Z dask.utilsr boolZuint8Z_get_dummies_dtype_defaultr%rHrKrrrr s4