U /e4@sddlZddlZddlZddlmZddlmZddlm Z dddZ ddd Z d d Z d d Z ddZddZddZddZddZddZdS)N)partial)islice)BagcCst|||d}|t|S)aChooses k unique random elements from a bag. Returns a new bag containing elements from the population while leaving the original population unchanged. Parameters ---------- population: Bag Elements to sample. k: integer, optional Number of elements to sample. split_every: int (optional) Group partitions into groups of this size while performing reduction. Defaults to 8. Examples -------- >>> import dask.bag as db >>> from dask.bag import random >>> b = db.from_sequence(range(5), npartitions=2) >>> list(random.sample(b, 3).compute()) # doctest: +SKIP [1, 3, 5]  populationk split_every)_samplemap_partitions_finalize_samplerrrresr3/tmp/pip-unpacked-wheel-dbjnr7gq/dask/bag/random.pysample srcCst|||d}|t|S)a7 Return a k sized list of elements chosen with replacement. Parameters ---------- population: Bag Elements to sample. k: integer, optional Number of elements to sample. split_every: int (optional) Group partitions into groups of this size while performing reduction. Defaults to 8. Examples -------- >>> import dask.bag as db >>> from dask.bag import random >>> b = db.from_sequence(range(5), npartitions=2) >>> list(random.choices(b, 3).compute()) # doctest: +SKIP [1, 1, 5] r)_sample_with_replacementr r r rrrchoices&src Csg}g}d}|D]4}|\}}||||7}t|} ||| fq||krZ|sZ||fSg} |D]*\}} | dkrb|| |} | | g| 7} qb|rtjnt} | || |d|fS)aq Reduce function used on the sample and choice functions. Parameters ---------- reduce_iter : iterable Each element is a tuple coming generated by the _sample_map_partitions function. replace: bool If True, sample with replacement. If False, sample without replacement. Returns a sequence of uniformly distributed samples; r)rweightsr)extendlenappendrndr&_weighted_sampling_without_replacement) reduce_iterrreplaceZns_kssniZs_iZn_iZk_ipZp_iZ sample_funcrrr_sample_reduce@s$     r cs4fddttD}fddt||DS)zk Source: Weighted random sampling with a reservoir, Pavlos S. Efraimidis, Paul G. Spirakis cs&g|]}tt||fqSr)mathlogrrandom).0r)rrr ksz:_weighted_sampling_without_replacement..csg|]}|dqS)rr)r$x)rrrr%ls)rangerheapqnlargest)rrreltr)rrrrfsrcCs4|dkrtd|jtt|dtt|ddt|dS)Nrz(Cannot take a negative number of samplesrFrrZout_typer) ValueError reductionr_sample_map_partitionsr rrrrrr os  r cCs |d}t||krtd|S)NrzSample larger than population)rr.)rrrrrrr zs r c Csgd}}t|}t||D]}|||d7}qttt|}|dt|}t ||D]N\}}||kr||t |<|ttt|9}|t|7}|d7}qf||fS)z Reservoir sampling strategy based on the L algorithm See https://en.wikipedia.org/wiki/Reservoir_sampling#An_optimal_algorithm rr) iterrrr!expr"rr# _geometric enumerate randrange) rr reservoir stream_lengthstreamewnxtrrrrr0s     r0cCs$|jtt|dtt|ddt|dS)Nr+Tr,r-)r/r'_sample_with_replacement_map_partitionsr rrrrrrs   rc st|}t|fddt|Dd}}ddt|D}dd|D}t|}t|dD]n\}||krt|D]D\} } | |krv|| <|| t9<|| t|| 7<qvt|}|d7}q^||fS)z Reservoir sampling with replacement, the main idea is to use k reservoirs of size 1 See Section Applications in http://utopia.duth.gr/~pefraimi/research/data/2007EncOfAlg.pdf csg|]}qSrrr$_r9rrr%sz;_sample_with_replacement_map_partitions..rcSsg|] }tqSr)rr#r=rrrr%scSsg|] }t|qSr)r3)r$Zwirrrr%s)r1nextr'minr4rr#r3) rrr8r6r7r:r;Zmin_nxtrjrrr?rr<s  r<cCs(tttddtd|dS)Nrr)intr!r"runiform)rrrrr3sr3)N)rN)r(r!r#r functoolsr itertoolsrZ dask.bag.corerrrr rr r r0rr<r3rrrrs     &