

pl.sync.to_iterable

Creates an iterable from a stage. Use this function to when you want to have more control over how the output stage is consumed, especifically, setting the maxsize argument can help you avoid OOM error if the consumer is slow.

Parameters:

Name	Type	Description	Default
`stage`	`Union[pypeln.sync.stage.Stage[~A], Iterable[~A], pypeln.utils.Undefined]`	A Stage or Iterable.	`<pypeln.utils.Undefined object at 0x7f27e00aaaf0>`
`maxsize`	`int`	This parameter is not used and only kept for API compatibility with the other modules.	`0`
`return_index`	`bool`	When set to `True` the resulting iterable will yield the `Elemen(index: Tuple[int, ...], value: Any)` which contains both the resulting value and the index parameter which holds information about the order of creation of the elements at the source.	`False`

Returns:

Type	Description
`Union[Iterable[~A], pypeln.utils.Partial[Iterable[~A]]]`	If the `stage` parameters is given then this function returns an iterable, else it returns a `Partial`.

Source code in pypeln/sync/api/to_iterable.py

def to_iterable(
    stage: tp.Union[
        Stage[A], tp.Iterable[A], pypeln_utils.Undefined
    ] = pypeln_utils.UNDEFINED,
    maxsize: int = 0,
    return_index: bool = False,
) -> tp.Union[tp.Iterable[A], pypeln_utils.Partial[tp.Iterable[A]]]:
    """
    Creates an iterable from a stage. Use this function to when you want to have more control over how the output stage is consumed, especifically, setting the `maxsize` argument can help you avoid OOM error if the consumer is slow.

    Arguments:
        stage: A Stage or Iterable.
        maxsize: This parameter is not used and only kept for API compatibility with the other modules.
        return_index: When set to `True` the resulting iterable will yield the `Elemen(index: Tuple[int, ...], value: Any)` which contains both the resulting value and the index parameter which holds information about the order of creation of the elements at the source.

    Returns:
        If the `stage` parameters is given then this function returns an iterable, else it returns a `Partial`.
    """

    if isinstance(stage, pypeln_utils.Undefined):
        return pypeln_utils.Partial(
            lambda stage: to_iterable(stage, maxsize=maxsize, return_index=return_index)
        )

    if isinstance(stage, Stage):
        iterable = stage.to_iterable(maxsize=maxsize, return_index=return_index)
    else:
        iterable = stage

    return iterable