U /e!@sdZddlZddlmZddlmZddl m Z e dZ e dZ e dZ e dZe dZe d Ze d Ze d Ze d Ze d Ze dZe dZe dZe dZddd ZddZdddZdd ZdddZd ddZdS)!z4 Dispatch in dask.dataframe. Also see extension.py N)Dispatchmake_meta_dispatch make_meta_obj meta_nonemptymeta_lib_from_arrayhash_object_dispatchgroup_split_dispatchget_parallel_typeZCategoricalDtypeconcattolistis_categorical_dtypeunion_categoricalsZgrouperpyarrow_schema_dispatchouterFTcKsFt|dkr|dStt|d}||f|||||d|SdS)aConcatenate, handling some edge cases: - Unions categoricals between partitions - Ignores empty partitions Parameters ---------- dfs : list of DataFrame, Series, or Index axis : int or str, optional join : str, optional uniform : bool, optional Whether to treat ``dfs[0]`` as representative of ``dfs[1:]``. Set to True if all arguments have the same columns and dtypes (but not necessarily categories). Default is False. ignore_index : bool, optional Whether to allow index values to be ignored/dropped during concatenation. Default is False. ignore_order : bool, optional Whether to ignore the order when doing the union of categoricals. Default is False. r)axisjoinuniformfilter_warning ignore_indexN)lenconcat_dispatchdispatchtype)dfsrrrrrkwargsfuncr;/tmp/pip-unpacked-wheel-dbjnr7gq/dask/dataframe/dispatch.pyr s cCs"t|d|}tt|}||S)NZdtype)getattris_categorical_dtype_dispatchrrobjrrrrr Js cCstt|}|||dS)N) categoriesordered)categorical_dtype_dispatchrr)metar#r$rrrrcategorical_dtypePsr'cCstt|}||S)N)tolist_dispatchrrr!rrrr UscCst|tjtjjtjjtjjt j fr*|j Szt ||dWSt k r|dk rntt|}|||dYSttj}|||dYSYnXdS)a This method creates meta-data based on the type of ``x``, and ``parent_meta`` if supplied. Parameters ---------- x : Object of any type. Object to construct meta-data from. index : Index, optional Any index to use in the metadata. This is a pass-through parameter to dispatches registered. parent_meta : Object, default None If ``x`` is of arbitrary types and thus Dask cannot determine which back-end to be used to generate the meta-data for this object type, in which case ``parent_meta`` will be used to determine which back-end to select and dispatch to. To use utilize this parameter ``make_meta_obj`` has be dispatched. If ``parent_meta`` is ``None``, a pandas DataFrame is used for ``parent_meta`` thats chooses pandas as the backend. Returns ------- A valid meta-data )indexN) isinstanceddZ_FramecoreZScalargroupbyZ_GroupByaccessorZAccessordaArrayZ_metar TypeErrorrrrpdZ DataFrame)xr)Z parent_metarrrr make_metaZs$  r4cCs tt|d}||||dS)Nr)sort_categories ignore_order)union_categoricals_dispatchrr)Zto_unionr5r6rrrrr s)rrFTF)NF)NN)FF)__doc__Zpandasr2Z dask.arrayarrayr/Zdask.dataframeZ dataframer+Z dask.utilsrrrrrrrr r%rr(r r7Zgrouper_dispatchrr r r'r r4r rrrrs:    -  3