pl.task.from_iterable

Creates a stage from an iterable. This function gives you more control of the iterable is consumed.

Parameters:

Name Type Description Default
iterable Union[Iterable[~T], AsyncIterable[~T], pypeln.utils.Undefined]

A source Iterable or AsyncIterable.

<pypeln.utils.Undefined object at 0x7f54c588db20>
use_thread bool

This parameter is not used and only kept for API compatibility with the other modules.

True

Returns:

Type Description
Union[pypeln.task.stage.Stage[~T], pypeln.utils.Partial[pypeln.task.stage.Stage[~T]]]

Returns a Stage if the iterable parameters is given, else it returns a Partial.

Source code in pypeln/task/api/from_iterable.py
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
def from_iterable(
    iterable: tp.Union[
        tp.Iterable[T], tp.AsyncIterable[T], pypeln_utils.Undefined
    ] = pypeln_utils.UNDEFINED,
    use_thread: bool = True,
) -> tp.Union[Stage[T], pypeln_utils.Partial[Stage[T]]]:
    """
    Creates a stage from an iterable. This function gives you more control of the iterable is consumed.
    Arguments:
        iterable: A source Iterable or AsyncIterable.
        use_thread: This parameter is not used and only kept for API compatibility with the other modules.

    Returns:
        Returns a `Stage` if the `iterable` parameters is given, else it returns a `Partial`.
    """

    if isinstance(iterable, pypeln_utils.Undefined):
        return pypeln_utils.Partial(
            lambda iterable: from_iterable(iterable, use_thread=use_thread)
        )

    return Stage(
        process_fn=FromIterable(iterable),
        workers=1,
        maxsize=0,
        total_sources=1,
        timeout=0,
        dependencies=[],
        on_start=None,
        on_done=None,
        f_args=[],
    )