Top

phi.builder module

The Builder extends the Expression class and adds methods that enable you to register external functions as methods. You should NOT register functions on the Builder or PythonBuilder classes, instead you should create a class of your own that extends these and register your functions there. The benefit of having this registration mechanism, instead of just implementing the methods yourself is that you can automate the process to register a collection of existing functions or methods.

See

"""
The `phi.builder.Builder` extends the [Expression](https://cgarciae.github.io/phi/dsl.m.html#phi.dsl.Expression) class and adds methods that enable you to register external functions as methods. You should NOT register functions on the `Builder` or [PythonBuilder](https://cgarciae.github.io/phi/python_builder.m.html) classes, instead you should create a class of your own that extends these and register your functions there. The benefit of having this registration mechanism, instead of just implementing the methods yourself is that you can automate the process to register a collection of existing functions or methods.

**See**

* `phi.builder.Builder.RegisterMethod`
* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.PatchAt`
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals


import inspect
from . import utils
from .utils import identity
import functools
from . import dsl

######################
# Helpers
######################

_True = lambda x: True
_False = lambda x: False
_None = lambda x: None
_NoLeadingUnderscore = lambda name: name[0] != "_"

#######################
### Builder
#######################

class Builder(dsl.Expression):
    """
    All the public methods of the `Builder`, `Expression` and `Expression` classes start with a capital letter on purpose to avoid name chashes with methods that you might register."""

    @classmethod
    def _RegisterMethod(cls, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True):
        if wrapped:
            f = functools.wraps(wrapped)(f)

        fn_signature = utils.get_method_sig(f)
        fn_docs = inspect.getdoc(f)
        name = alias if alias else f.__name__
        original_name = f.__name__ if wrapped else original_name if original_name else name

        f.__name__ = str(name)
        f.__doc__ = doc if doc else ("""
THIS METHOD IS AUTOMATICALLY GENERATED

    {builder_class}.{name}(*args, **kwargs)

It accepts the same arguments as `{library_path}{original_name}`. """ + explanation + """

**{library_path}{original_name}**

    {fn_docs}

        """).format(original_name=original_name, name=name, fn_docs=fn_docs, library_path=library_path, builder_class=cls.__name__) if explain else fn_docs

        if name in cls.__core__:
            raise Exception("Can't add method '{0}' because its on __core__".format(name))

        f = method_type(f)
        setattr(cls, name, f)


    @classmethod
    def RegisterMethod(cls, *args, **kwargs):
        """
**RegisterMethod**

    RegisterMethod(f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True)

`classmethod` for registering functions as methods of this class.

**Arguments**

* **f** : the particular function being registered as a method
* **library_path** : library from where `f` comes from, unless you pass an empty string, put a period `"."` at the end of the library name.
* `alias=None` : alias for the name/method being registered
* `original_name=None` : name of the original function, used for documentation purposes.
* `doc=None` : complete documentation of the method being registered
* `wrapped=None` : if you are registering a function which wraps around another function, pass this other function through `wrapped` to get better documentation, this is specially useful is you register a bunch of functions in a for loop. Please include an `explanation` to tell how the actual function differs from the wrapped one.
* `explanation=""` : especify any additional information for the documentation of the method being registered, you can use any of the following format tags within this string and they will be replace latter on: `{original_name}`, `{name}`, `{fn_docs}`, `{library_path}`, `{builder_class}`.
* `method_type=identity` : by default its applied but does nothing, you might also want to register functions as `property`, `classmethod`, `staticmethod`
* `explain=True` : decide whether or not to show any kind of explanation, its useful to set it to `False` if you are using a `Register*` decorator and will only use the function as a registered method.

A main feature of `phi` is that it enables you to integrate your library or even an existing library with the DSL. You can achieve three levels of integration

1. Passing your functions to the DSL. This a very general machanism -since you could actually do everything with python lamdas- but in practice functions often receive multiple parameters.
2. Creating partials with the `Then*` method family. Using this you could integrate any function, but it will add a lot of noise if you use heavily on it.
3. Registering functions as methods of a `Builder` derived class. This produces the most readable code and its the approach you should take if you want to create a Phi-based library or a helper class.

While point 3 is the most desirable it has a cost: you need to create your own `phi.builder.Builder`-derived class. This is because SHOULD NOT register functions to existing builders e.g. the `phi.builder.Builder` or [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) provided by phi because that would pollute the `P` object. Instead you should create a custom class that derives from `phi.builder.Builder`,  [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) or another custom builder depending on your needs and register your functions to that class.

**Examples**

Say you have a function on a library called `"my_lib"`

    def some_fun(obj, arg1, arg2):
        # code

You could use it with the dsl like this

    from phi import P, Then

    P.Pipe(
        input,
        ...
        Then(some_fun, arg1, arg2)
        ...
    )

assuming the first parameter `obj` is being piped down. However if you do this very often or you are creating a library, you are better off creating a custom class derived from `Builder` or `PythonBuilder`

    from phi import Builder #or PythonBuilder

    class MyBuilder(Builder): # or PythonBuilder
        pass

and registering your function as a method. The first way you could do this is by creating a wrapper function for `some_fun` and registering it as a method

    def some_fun_wrapper(self, arg1, arg2):
        return self.Then(some_fun, arg1, arg2)

    MyBuilder.RegisterMethod(some_fun_wrapper, "my_lib.", wrapped=some_fun)

Here we basically created a shortcut for the original expression `Then(some_fun, arg1, arg2)`. You could also do this using a decorator

    @MyBuilder.RegisterMethod("my_lib.", wrapped=some_fun)
    def some_fun_wrapper(self, arg1, arg2):
        return self.Then(some_fun, arg1, arg2)

However, this is such a common task that we've created the method `Register` to avoid you from having to create the wrapper. With it you could register the function `some_fun` directly as a method like this

    MyBuilder.Register(some_fun, "my_lib.")

or by using a decorator over the original function definition

    @MyBuilder.Register("my_lib.")
    def some_fun(obj, arg1, arg2):
        # code

Once done you've done any of the previous approaches you can create a custom global object e.g. `M` and use it instead of/along with `P`

    M = MyBuilder(lambda x: x)

    M.Pipe(
        input,
        ...
        M.some_fun(arg1, args)
        ...
    )

**Argument position**

`phi.builder.Builder.Register` internally uses `phi.builder.Builder.Then`, this is only useful if the object being piped is intended to be passed as the first argument of the function being registered, if this is not the case you could use `phi.builder.Builder.Register2`, `phi.builder.Builder.Register3`, ..., `phi.builder.Builder.Register5` or `phi.builder.Builder.RegisterAt` to set an arbitrary position, these functions will internally use `phi.builder.Builder.Then2`, `phi.builder.Builder.Then3`, ..., `phi.builder.Builder.Then5` or `phi.builder.Builder.ThenAt` respectively.

**Wrapping functions**

Sometimes you have an existing function that you would like to modify slightly so it plays nicely with the DSL, what you normally do is create a function that wraps around it and passes the arguments to it in a way that is convenient

    import some_lib

    @MyBuilder.Register("some_lib.")
    def some_fun(a, n):
        return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified

When you do this -as a side effect- you loose the original documentation, to avoid this you can use the Registers `wrapped` argument along with the `explanation` argument to clarity the situation

    import some_lib

    some_fun_explanation = "However, it differs in that `n` is automatically subtracted `1`"

    @MyBuilder.Register("some_lib.", wrapped=some_lib.some_fun, explanation=some_fun_explanation)
    def some_fun(a, n):
        return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified

Now the documentation for `MyBuilder.some_fun` will be a little bit nicer since it includes the original documentation from `some_lib.some_fun`. This behaviour is specially useful if you are wrapping an entire 3rd party library, you usually automate the process iterating over all the funcitions in a for loop. The `phi.builder.Builder.PatchAt` method lets you register and entire module using a few lines of code, however, something you have to do thing more manually and do the iteration yourself.

**See Also**

* `phi.builder.Builder.PatchAt`
* `phi.builder.Builder.RegisterAt`
        """
        unpack_error = True

        try:
            f, library_path = args
            unpack_error = False
            cls._RegisterMethod(f, library_path, **kwargs)

        except:
            if not unpack_error:
                raise

            def register_decorator(f):
                library_path, = args
                cls._RegisterMethod(f, library_path, **kwargs)

                return f
            return register_decorator


    @classmethod
    def _RegisterAt(cls, n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None):

        _wrapped = wrapped if wrapped else f

        try:
            @functools.wraps(f)
            def method(self, *args, **kwargs):

                kwargs['_return_type'] = _return_type
                return self.ThenAt(n, f, *args, **kwargs)
        except:
            raise

        all_args, previous_args, last_arg = _make_args_strs(n)

        explanation = """
However, the 1st argument is omitted, a partial with the rest of the arguments is returned which expects the 1st argument such that

    {library_path}{original_name}("""+ all_args +"""*args, **kwargs)

is equivalent to

    {builder_class}.{name}("""+ previous_args +"""*args, **kwargs)("""+ last_arg +""")

        """ + explanation  if explain else ""

        cls.RegisterMethod(method, library_path, alias=alias, original_name=original_name, doc=doc, wrapped=wrapped, explanation=explanation, method_type=method_type, explain=explain)

    @classmethod
    def RegisterAt(cls, *args, **kwargs):
        """
**RegisterAt**

    RegisterAt(n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None)

Most of the time you don't want to register an method as such, that is, you don't care about the `self` builder object, instead you want to register a function that transforms the value being piped down the DSL. For this you can use `RegisterAt` so e.g.

    def some_fun(obj, arg1, arg2):
        # code

    @MyBuilder.RegisterMethod("my_lib.")
    def some_fun_wrapper(self, arg1, arg2):
        return self.ThenAt(1, some_fun, arg1, arg2)

can be written directly as

    @MyBuilder.RegisterAt(1, "my_lib.")
    def some_fun(obj, arg1, arg2):
        # code

For this case you can just use `Register` which is a shortcut for `RegisterAt(1, ...)`

    @MyBuilder.Register("my_lib.")
    def some_fun(obj, arg1, arg2):
        # code

**Also See**

* `phi.builder.Builder.RegisterMethod`
        """
        unpack_error = True

        try:
            n, f, library_path = args
            unpack_error = False
            cls._RegisterAt(n, f, library_path, **kwargs)

        except:
            if not unpack_error:
                raise

            def register_decorator(f):
                n, library_path = args
                cls._RegisterAt(n, f, library_path, **kwargs)

                return f
            return register_decorator

    @classmethod
    def Register0(cls, *args, **kwargs):
        """
`Register0(...)` is a shortcut for `RegisterAt(0, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(0, *args, **kwargs)

    @classmethod
    def Register(cls, *args, **kwargs):
        """
`Register(...)` is a shortcut for `RegisterAt(1, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(1, *args, **kwargs)

    @classmethod
    def Register2(cls, *args, **kwargs):
        """
`Register2(...)` is a shortcut for `RegisterAt(2, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(2, *args, **kwargs)

    @classmethod
    def Register3(cls, *args, **kwargs):
        """
`Register3(...)` is a shortcut for `RegisterAt(3, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(3, *args, **kwargs)

    @classmethod
    def Register4(cls, *args, **kwargs):
        """
`Register4(...)` is a shortcut for `RegisterAt(4, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(4, *args, **kwargs)

    @classmethod
    def Register5(cls, *args, **kwargs):
        """
`Register5(...)` is a shortcut for `RegisterAt(5, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(5, *args, **kwargs)

    @classmethod
    def PatchAt(cls, n, module, method_wrapper=None, module_alias=None, method_name_modifier=utils.identity, blacklist_predicate=_False, whitelist_predicate=_True, return_type_predicate=_None, getmembers_predicate=inspect.isfunction, admit_private=False, explanation=""):
        """
This classmethod lets you easily patch all of functions/callables from a module or class as methods a Builder class.

**Arguments**

* **n** : the position the the object being piped will take in the arguments when the function being patched is applied. See `RegisterMethod` and `ThenAt`.
* **module** : a module or class from which the functions/methods/callables will be taken.
* `module_alias = None` : an optional alias for the module used for documentation purposes.
* `method_name_modifier = lambda f_name: None` : a function that can modify the name of the method will take. If `None` the name of the function will be used.
* `blacklist_predicate = lambda f_name: name[0] != "_"` : A predicate that determines which functions are banned given their name. By default it excludes all function whose name start with `'_'`. `blacklist_predicate` can also be of type list, in which case all names contained in this list will be banned.
* `whitelist_predicate = lambda f_name: True` : A predicate that determines which functions are admitted given their name. By default it include any function. `whitelist_predicate` can also be of type list, in which case only names contained in this list will be admitted. You can use both `blacklist_predicate` and `whitelist_predicate` at the same time.
* `return_type_predicate = lambda f_name: None` : a predicate that determines the `_return_type` of the Builder. By default it will always return `None`. See `phi.builder.Builder.ThenAt`.
* `getmembers_predicate = inspect.isfunction` : a predicate that determines what type of elements/members will be fetched by the `inspect` module, defaults to [inspect.isfunction](https://docs.python.org/2/library/inspect.html#inspect.isfunction). See [getmembers](https://docs.python.org/2/library/inspect.html#inspect.getmembers).

**Examples**

Lets patch ALL the main functions from numpy into a custom builder!

    from phi import PythonBuilder #or Builder
    import numpy as np

    class NumpyBuilder(PythonBuilder): #or Builder
        "A Builder for numpy functions!"
        pass

    NumpyBuilder.PatchAt(1, np)

    N = NumpyBuilder(lambda x: x)

Thats it! Although a serious patch would involve filtering out functions that don't take arrays. Another common task would be to use `NumpyBuilder.PatchAt(2, ...)` (`PatchAt(n, ..)` in general) when convenient to send the object being pipe to the relevant argument of the function. The previous is usually done with and a combination of `whitelist_predicate`s and `blacklist_predicate`s on `PatchAt(1, ...)` and `PatchAt(2, ...)` to filter or include the approriate functions on each kind of patch. Given the previous code we could now do

    import numpy as np

    x = np.array([[1,2],[3,4]])
    y = np.array([[5,6],[7,8]])

    z = N.Pipe(
        x, N
        .dot(y)
        .add(x)
        .transpose()
        .sum(axis=1)
    )

Which is strictly equivalent to

    import numpy as np

    x = np.array([[1,2],[3,4]])
    y = np.array([[5,6],[7,8]])

    z = np.dot(x, y)
    z = np.add(z, x)
    z = np.transpose(z)
    z = np.sum(z, axis=1)

The thing to notice is that with the `NumpyBuilder` we avoid the repetitive and needless passing and reassigment of the `z` variable, this removes a lot of noise from our code.
        """
        _rtp = return_type_predicate

        return_type_predicate = (lambda x: _rtp) if inspect.isclass(_rtp) and issubclass(_rtp, Builder) else _rtp
        module_name = module_alias if module_alias else module.__name__ + '.'
        patch_members = _get_patch_members(module, blacklist_predicate=blacklist_predicate, whitelist_predicate=whitelist_predicate, getmembers_predicate=getmembers_predicate, admit_private=admit_private)

        if type(whitelist_predicate) is list and "convolution2d" in whitelist_predicate:
            import ipdb
            #ipdb.set_trace()

        for name, f in patch_members:
            wrapped = None

            if method_wrapper:
                g = method_wrapper(f)
                wrapped = f
            else:
                g = f

            cls.RegisterAt(n, g, module_name, wrapped=wrapped, _return_type=return_type_predicate(name), alias=method_name_modifier(name), explanation=explanation)


Builder.__core__ = [ name for name, f in inspect.getmembers(Builder, inspect.ismethod) ]



#######################
# Helper functions
#######################

def _make_args_strs(n):

    if n == 0:
        return "", "", "x"

    n += 1
    all_args = [ 'x' + str(i) for i in range(1, n) ]
    last = all_args[n-2]
    previous = all_args[:n-2]

    return ", ".join(all_args + [""]), ", ".join(previous + [""]), last

def _get_patch_members(module, blacklist_predicate=_NoLeadingUnderscore, whitelist_predicate=_True, _return_type=None, getmembers_predicate=inspect.isfunction, admit_private=False):

    if type(whitelist_predicate) is list:
        whitelist = whitelist_predicate
        whitelist_predicate = lambda x: x in whitelist

    if type(blacklist_predicate) is list:
        blacklist = blacklist_predicate
        blacklist_predicate = lambda x: x in blacklist or '_' == x[0] if not admit_private else False

    return [
        (name, f) for (name, f) in inspect.getmembers(module, getmembers_predicate) if whitelist_predicate(name) and not blacklist_predicate(name)
    ]

Module variables

var name

Classes

class Builder

All the public methods of the Builder, Expression and Expression classes start with a capital letter on purpose to avoid name chashes with methods that you might register.

class Builder(dsl.Expression):
    """
    All the public methods of the `Builder`, `Expression` and `Expression` classes start with a capital letter on purpose to avoid name chashes with methods that you might register."""

    @classmethod
    def _RegisterMethod(cls, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True):
        if wrapped:
            f = functools.wraps(wrapped)(f)

        fn_signature = utils.get_method_sig(f)
        fn_docs = inspect.getdoc(f)
        name = alias if alias else f.__name__
        original_name = f.__name__ if wrapped else original_name if original_name else name

        f.__name__ = str(name)
        f.__doc__ = doc if doc else ("""
THIS METHOD IS AUTOMATICALLY GENERATED

    {builder_class}.{name}(*args, **kwargs)

It accepts the same arguments as `{library_path}{original_name}`. """ + explanation + """

**{library_path}{original_name}**

    {fn_docs}

        """).format(original_name=original_name, name=name, fn_docs=fn_docs, library_path=library_path, builder_class=cls.__name__) if explain else fn_docs

        if name in cls.__core__:
            raise Exception("Can't add method '{0}' because its on __core__".format(name))

        f = method_type(f)
        setattr(cls, name, f)


    @classmethod
    def RegisterMethod(cls, *args, **kwargs):
        """
**RegisterMethod**

    RegisterMethod(f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True)

`classmethod` for registering functions as methods of this class.

**Arguments**

* **f** : the particular function being registered as a method
* **library_path** : library from where `f` comes from, unless you pass an empty string, put a period `"."` at the end of the library name.
* `alias=None` : alias for the name/method being registered
* `original_name=None` : name of the original function, used for documentation purposes.
* `doc=None` : complete documentation of the method being registered
* `wrapped=None` : if you are registering a function which wraps around another function, pass this other function through `wrapped` to get better documentation, this is specially useful is you register a bunch of functions in a for loop. Please include an `explanation` to tell how the actual function differs from the wrapped one.
* `explanation=""` : especify any additional information for the documentation of the method being registered, you can use any of the following format tags within this string and they will be replace latter on: `{original_name}`, `{name}`, `{fn_docs}`, `{library_path}`, `{builder_class}`.
* `method_type=identity` : by default its applied but does nothing, you might also want to register functions as `property`, `classmethod`, `staticmethod`
* `explain=True` : decide whether or not to show any kind of explanation, its useful to set it to `False` if you are using a `Register*` decorator and will only use the function as a registered method.

A main feature of `phi` is that it enables you to integrate your library or even an existing library with the DSL. You can achieve three levels of integration

1. Passing your functions to the DSL. This a very general machanism -since you could actually do everything with python lamdas- but in practice functions often receive multiple parameters.
2. Creating partials with the `Then*` method family. Using this you could integrate any function, but it will add a lot of noise if you use heavily on it.
3. Registering functions as methods of a `Builder` derived class. This produces the most readable code and its the approach you should take if you want to create a Phi-based library or a helper class.

While point 3 is the most desirable it has a cost: you need to create your own `phi.builder.Builder`-derived class. This is because SHOULD NOT register functions to existing builders e.g. the `phi.builder.Builder` or [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) provided by phi because that would pollute the `P` object. Instead you should create a custom class that derives from `phi.builder.Builder`,  [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) or another custom builder depending on your needs and register your functions to that class.

**Examples**

Say you have a function on a library called `"my_lib"`

    def some_fun(obj, arg1, arg2):
        # code

You could use it with the dsl like this

    from phi import P, Then

    P.Pipe(
        input,
        ...
        Then(some_fun, arg1, arg2)
        ...
    )

assuming the first parameter `obj` is being piped down. However if you do this very often or you are creating a library, you are better off creating a custom class derived from `Builder` or `PythonBuilder`

    from phi import Builder #or PythonBuilder

    class MyBuilder(Builder): # or PythonBuilder
        pass

and registering your function as a method. The first way you could do this is by creating a wrapper function for `some_fun` and registering it as a method

    def some_fun_wrapper(self, arg1, arg2):
        return self.Then(some_fun, arg1, arg2)

    MyBuilder.RegisterMethod(some_fun_wrapper, "my_lib.", wrapped=some_fun)

Here we basically created a shortcut for the original expression `Then(some_fun, arg1, arg2)`. You could also do this using a decorator

    @MyBuilder.RegisterMethod("my_lib.", wrapped=some_fun)
    def some_fun_wrapper(self, arg1, arg2):
        return self.Then(some_fun, arg1, arg2)

However, this is such a common task that we've created the method `Register` to avoid you from having to create the wrapper. With it you could register the function `some_fun` directly as a method like this

    MyBuilder.Register(some_fun, "my_lib.")

or by using a decorator over the original function definition

    @MyBuilder.Register("my_lib.")
    def some_fun(obj, arg1, arg2):
        # code

Once done you've done any of the previous approaches you can create a custom global object e.g. `M` and use it instead of/along with `P`

    M = MyBuilder(lambda x: x)

    M.Pipe(
        input,
        ...
        M.some_fun(arg1, args)
        ...
    )

**Argument position**

`phi.builder.Builder.Register` internally uses `phi.builder.Builder.Then`, this is only useful if the object being piped is intended to be passed as the first argument of the function being registered, if this is not the case you could use `phi.builder.Builder.Register2`, `phi.builder.Builder.Register3`, ..., `phi.builder.Builder.Register5` or `phi.builder.Builder.RegisterAt` to set an arbitrary position, these functions will internally use `phi.builder.Builder.Then2`, `phi.builder.Builder.Then3`, ..., `phi.builder.Builder.Then5` or `phi.builder.Builder.ThenAt` respectively.

**Wrapping functions**

Sometimes you have an existing function that you would like to modify slightly so it plays nicely with the DSL, what you normally do is create a function that wraps around it and passes the arguments to it in a way that is convenient

    import some_lib

    @MyBuilder.Register("some_lib.")
    def some_fun(a, n):
        return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified

When you do this -as a side effect- you loose the original documentation, to avoid this you can use the Registers `wrapped` argument along with the `explanation` argument to clarity the situation

    import some_lib

    some_fun_explanation = "However, it differs in that `n` is automatically subtracted `1`"

    @MyBuilder.Register("some_lib.", wrapped=some_lib.some_fun, explanation=some_fun_explanation)
    def some_fun(a, n):
        return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified

Now the documentation for `MyBuilder.some_fun` will be a little bit nicer since it includes the original documentation from `some_lib.some_fun`. This behaviour is specially useful if you are wrapping an entire 3rd party library, you usually automate the process iterating over all the funcitions in a for loop. The `phi.builder.Builder.PatchAt` method lets you register and entire module using a few lines of code, however, something you have to do thing more manually and do the iteration yourself.

**See Also**

* `phi.builder.Builder.PatchAt`
* `phi.builder.Builder.RegisterAt`
        """
        unpack_error = True

        try:
            f, library_path = args
            unpack_error = False
            cls._RegisterMethod(f, library_path, **kwargs)

        except:
            if not unpack_error:
                raise

            def register_decorator(f):
                library_path, = args
                cls._RegisterMethod(f, library_path, **kwargs)

                return f
            return register_decorator


    @classmethod
    def _RegisterAt(cls, n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None):

        _wrapped = wrapped if wrapped else f

        try:
            @functools.wraps(f)
            def method(self, *args, **kwargs):

                kwargs['_return_type'] = _return_type
                return self.ThenAt(n, f, *args, **kwargs)
        except:
            raise

        all_args, previous_args, last_arg = _make_args_strs(n)

        explanation = """
However, the 1st argument is omitted, a partial with the rest of the arguments is returned which expects the 1st argument such that

    {library_path}{original_name}("""+ all_args +"""*args, **kwargs)

is equivalent to

    {builder_class}.{name}("""+ previous_args +"""*args, **kwargs)("""+ last_arg +""")

        """ + explanation  if explain else ""

        cls.RegisterMethod(method, library_path, alias=alias, original_name=original_name, doc=doc, wrapped=wrapped, explanation=explanation, method_type=method_type, explain=explain)

    @classmethod
    def RegisterAt(cls, *args, **kwargs):
        """
**RegisterAt**

    RegisterAt(n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None)

Most of the time you don't want to register an method as such, that is, you don't care about the `self` builder object, instead you want to register a function that transforms the value being piped down the DSL. For this you can use `RegisterAt` so e.g.

    def some_fun(obj, arg1, arg2):
        # code

    @MyBuilder.RegisterMethod("my_lib.")
    def some_fun_wrapper(self, arg1, arg2):
        return self.ThenAt(1, some_fun, arg1, arg2)

can be written directly as

    @MyBuilder.RegisterAt(1, "my_lib.")
    def some_fun(obj, arg1, arg2):
        # code

For this case you can just use `Register` which is a shortcut for `RegisterAt(1, ...)`

    @MyBuilder.Register("my_lib.")
    def some_fun(obj, arg1, arg2):
        # code

**Also See**

* `phi.builder.Builder.RegisterMethod`
        """
        unpack_error = True

        try:
            n, f, library_path = args
            unpack_error = False
            cls._RegisterAt(n, f, library_path, **kwargs)

        except:
            if not unpack_error:
                raise

            def register_decorator(f):
                n, library_path = args
                cls._RegisterAt(n, f, library_path, **kwargs)

                return f
            return register_decorator

    @classmethod
    def Register0(cls, *args, **kwargs):
        """
`Register0(...)` is a shortcut for `RegisterAt(0, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(0, *args, **kwargs)

    @classmethod
    def Register(cls, *args, **kwargs):
        """
`Register(...)` is a shortcut for `RegisterAt(1, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(1, *args, **kwargs)

    @classmethod
    def Register2(cls, *args, **kwargs):
        """
`Register2(...)` is a shortcut for `RegisterAt(2, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(2, *args, **kwargs)

    @classmethod
    def Register3(cls, *args, **kwargs):
        """
`Register3(...)` is a shortcut for `RegisterAt(3, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(3, *args, **kwargs)

    @classmethod
    def Register4(cls, *args, **kwargs):
        """
`Register4(...)` is a shortcut for `RegisterAt(4, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(4, *args, **kwargs)

    @classmethod
    def Register5(cls, *args, **kwargs):
        """
`Register5(...)` is a shortcut for `RegisterAt(5, ...)`

**Also See**

* `phi.builder.Builder.RegisterAt`
* `phi.builder.Builder.RegisterMethod`
        """
        return cls.RegisterAt(5, *args, **kwargs)

    @classmethod
    def PatchAt(cls, n, module, method_wrapper=None, module_alias=None, method_name_modifier=utils.identity, blacklist_predicate=_False, whitelist_predicate=_True, return_type_predicate=_None, getmembers_predicate=inspect.isfunction, admit_private=False, explanation=""):
        """
This classmethod lets you easily patch all of functions/callables from a module or class as methods a Builder class.

**Arguments**

* **n** : the position the the object being piped will take in the arguments when the function being patched is applied. See `RegisterMethod` and `ThenAt`.
* **module** : a module or class from which the functions/methods/callables will be taken.
* `module_alias = None` : an optional alias for the module used for documentation purposes.
* `method_name_modifier = lambda f_name: None` : a function that can modify the name of the method will take. If `None` the name of the function will be used.
* `blacklist_predicate = lambda f_name: name[0] != "_"` : A predicate that determines which functions are banned given their name. By default it excludes all function whose name start with `'_'`. `blacklist_predicate` can also be of type list, in which case all names contained in this list will be banned.
* `whitelist_predicate = lambda f_name: True` : A predicate that determines which functions are admitted given their name. By default it include any function. `whitelist_predicate` can also be of type list, in which case only names contained in this list will be admitted. You can use both `blacklist_predicate` and `whitelist_predicate` at the same time.
* `return_type_predicate = lambda f_name: None` : a predicate that determines the `_return_type` of the Builder. By default it will always return `None`. See `phi.builder.Builder.ThenAt`.
* `getmembers_predicate = inspect.isfunction` : a predicate that determines what type of elements/members will be fetched by the `inspect` module, defaults to [inspect.isfunction](https://docs.python.org/2/library/inspect.html#inspect.isfunction). See [getmembers](https://docs.python.org/2/library/inspect.html#inspect.getmembers).

**Examples**

Lets patch ALL the main functions from numpy into a custom builder!

    from phi import PythonBuilder #or Builder
    import numpy as np

    class NumpyBuilder(PythonBuilder): #or Builder
        "A Builder for numpy functions!"
        pass

    NumpyBuilder.PatchAt(1, np)

    N = NumpyBuilder(lambda x: x)

Thats it! Although a serious patch would involve filtering out functions that don't take arrays. Another common task would be to use `NumpyBuilder.PatchAt(2, ...)` (`PatchAt(n, ..)` in general) when convenient to send the object being pipe to the relevant argument of the function. The previous is usually done with and a combination of `whitelist_predicate`s and `blacklist_predicate`s on `PatchAt(1, ...)` and `PatchAt(2, ...)` to filter or include the approriate functions on each kind of patch. Given the previous code we could now do

    import numpy as np

    x = np.array([[1,2],[3,4]])
    y = np.array([[5,6],[7,8]])

    z = N.Pipe(
        x, N
        .dot(y)
        .add(x)
        .transpose()
        .sum(axis=1)
    )

Which is strictly equivalent to

    import numpy as np

    x = np.array([[1,2],[3,4]])
    y = np.array([[5,6],[7,8]])

    z = np.dot(x, y)
    z = np.add(z, x)
    z = np.transpose(z)
    z = np.sum(z, axis=1)

The thing to notice is that with the `NumpyBuilder` we avoid the repetitive and needless passing and reassigment of the `z` variable, this removes a lot of noise from our code.
        """
        _rtp = return_type_predicate

        return_type_predicate = (lambda x: _rtp) if inspect.isclass(_rtp) and issubclass(_rtp, Builder) else _rtp
        module_name = module_alias if module_alias else module.__name__ + '.'
        patch_members = _get_patch_members(module, blacklist_predicate=blacklist_predicate, whitelist_predicate=whitelist_predicate, getmembers_predicate=getmembers_predicate, admit_private=admit_private)

        if type(whitelist_predicate) is list and "convolution2d" in whitelist_predicate:
            import ipdb
            #ipdb.set_trace()

        for name, f in patch_members:
            wrapped = None

            if method_wrapper:
                g = method_wrapper(f)
                wrapped = f
            else:
                g = f

            cls.RegisterAt(n, g, module_name, wrapped=wrapped, _return_type=return_type_predicate(name), alias=method_name_modifier(name), explanation=explanation)

Ancestors (in MRO)

  • Builder
  • phi.dsl.Expression
  • __builtin__.object

Static methods

def Context(

*args)

Builder Core. Also available as a global function as phi.Context.

Returns the context object of the current dsl.With statemente.

Arguments

  • *args: By design Context accepts any number of arguments and completely ignores them.

This is a classmethod and it doesnt return a Builder/Expression by design so it can be called directly:

from phi import P, Context, Obj

def read_file(z):
    f = Context()
    return f.read()

lines = P.Pipe(
    "text.txt",
    P.With( open,
        read_file,
        Obj.split("\n")
    )
)

Here we called Context with no arguments to get the context back, however, since you can also give this function an argument (which it will ignore) it can be passed to the DSL so we can rewrite the previous as:

from phi import P, Context, Obj

lines = P.Pipe(
    "text.txt",
    P.With( open,
        Context, # f
        Obj.read()
        Obj.split("\n")
    )
)

Context yields an exception when used outside of a With block.

Also see

@staticmethod
def Context(*args):
    """
ilder Core**. Also available as a global function as `phi.Context`.
rns the context object of the current `dsl.With` statemente.
guments**
*args**: By design `Context` accepts any number of arguments and completely ignores them.
 is a classmethod and it doesnt return a `Builder`/`Expression` by design so it can be called directly:
from phi import P, Context, Obj
def read_file(z):
    f = Context()
    return f.read()
lines = P.Pipe(
    "text.txt",
    P.With( open,
        read_file,
        Obj.split("\\n")
    )
)
 we called `Context` with no arguments to get the context back, however, since you can also give this function an argument (which it will ignore) it can be passed to the DSL so we can rewrite the previous as:
from phi import P, Context, Obj
lines = P.Pipe(
    "text.txt",
    P.With( open,
        Context, # f
        Obj.read()
        Obj.split("\\n")
    )
)
text` yields an exception when used outside of a `With` block.
so see**
hi.builder.Builder.Obj`
sl](https://cgarciae.github.io/phi/dsl.m.html)
    """
    if _WithContextManager.WITH_GLOBAL_CONTEXT is utils.NO_VALUE:
        raise Exception("Cannot use 'Context' outside of a 'With' block")
    return _WithContextManager.WITH_GLOBAL_CONTEXT

Instance variables

var Obj

Obj is a property that returns an object that defines the __getattr__ method which when called helps you create a partial that emulates a method call. The following expression

Obj.some_method(x1, x2, ...)

is equivalent to

lambda obj: obj.some_method(x1, x2, ...)

Examples

from phi import P, Obj

assert "hello world" == P.Pipe(
    "  HELLO HELLO {0}     ",
    Obj.format("WORLD"),  # "   HELLO HELLO WORLD     "
    Obj.strip(),          # "HELLO HELLO WORLD"
    Obj.lower()           # "hello hello world"
    Obj.split(' ')        # ["hello", "hello", "world"]
    Obj.count("hello")    # 2
)

Also see

var Read

Giving names and saving parts of your computation to use then latter is useful to say the least. In Phi the expression

Write(x = expr)

creates a reference x given the value of expr which you can call latter. To read the previous you would use any of the following expressions

Read('x')
Read.x

Example

Lets see a common situation where you would use this

from phi import P, List, Seq, Read, Write

result = P.Pipe(
    input,
    Write(ref = f1),
    f2,
    List(
        f3
    ,
        Seq(
            Read('ref'),
            f4
        )
    )
)

Here you save the value outputed by fun_1 and the load it as the initial value of the second branch. In normal python the previous would be almost equivalent to

x = f1(input)
ref = x
x = f2(x)

result = [
    f3(x)
,
    f4(ref)
]

var Rec

Rec is a property that returns an object that defines the __getattr__ and __getitem__ methods which when called help you create lambdas that emulates a field access. The following expression

Rec.some_field

is equivalent to

lambda rec: rec.some_field

Examples

from phi import P, Obj, Rec

class Point(object):

    def __init__(self, x, y):
        self.x = x
        self.y = y

    def flip_cords(self):
        y = self.y
        self.y = self.x
        self.x = y

assert 4 == P.Pipe(
    Point(1, 2),         # point(x=1, y=2)
    Obj.flip_cords(),    # point(x=2, y=1)
    Rec.x,               # point.x = 2
    P * 2                # 2 * 2 = 4
)

Also see

var Ref

Returns an object that helps you to inmediatly create and read references.

Creating Refences

You can manually create a Ref outside the DSL using Ref and then pass to as/to a Read or Write expression. Here is a contrived example

from phi import P

r = P.Ref('r')

assert [600, 3, 6] == P.Pipe(
    2,
    P + 1, {'a'},  # a = 2 + 1 = 3
    P * 2, {'b'},  # b = 3 * 2 = 6
    P * 100, {'c', r },  # c = r = 6 * 100 = 600
    ['c', 'a', 'b']
)

assert r() == 600

Reading Refences from the Current Context

While the expression Read.a with return a function that will discard its argument and return the value of the reference x in the current context, the expression Ref.x will return the value inmediatly, this is useful when using it inside pyton lambdas.

Read.x(None) <=> Ref.x

As an example

from phi import P, Obj, Ref

assert {'a': 97, 'b': 98, 'c': 99} == P.Pipe(
    "a b c", Obj
    .split(' ').Write.keys  # keys = ['a', 'b', 'c']
    .map(ord),  # [ord('a'), ord('b'), ord('c')] == [97, 98, 99]
    lambda it: zip(Ref.keys, it),  # [('a', 97), ('b', 98), ('c', 99)]
    dict   # {'a': 97, 'b': 98, 'c': 99}
)

Methods

def __init__(

self, f=<function state_identity at 0x7fea7e92eaa0>)

def __init__(self, f=utils.state_identity):
    self._f = f

def Dict(

self, **branches)

def Dict(self, **branches):
    gs = { key : _parse(value)._f for key, value in branches.items() }
    def h(x, state):
        ys = {}
        for key, g in gs.items():
            y, state = g(x, state)
            ys[key] = y
        return _RecordObject(**ys), state
    return self.__then__(h)

def Elif(

self, condition, *then, **kwargs)

See phi.dsl.Expression.If

def Elif(self, condition, *then, **kwargs):
    """See `phi.dsl.Expression.If`"""
    root = self._root
    ast = self._ast
    cond_f = _parse(condition)._f
    then_f = E.Seq(*then)._f
    else_f = utils.state_identity
    next_else = (cond_f, then_f, else_f)
    ast = _add_else(ast, next_else)
    g = _compile_if(ast)
    expr = root.__then__(g, **kwargs)
    expr._ast = ast
    expr._root = root
    return expr

def Else(

self, *Else, **kwargs)

See phi.dsl.Expression.If

def Else(self, *Else, **kwargs):
    """See `phi.dsl.Expression.If`"""
    root = self._root
    ast = self._ast
    next_else = E.Seq(*Else)._f
    ast = _add_else(ast, next_else)
    g = _compile_if(ast)
    return root.__then__(g, **kwargs)

def F(

self, expr)

def F(self, expr):
    return self >> expr

def If(

self, condition, *then, **kwargs)

If

If(Predicate, *Then)

Having conditionals expressions a necesity in every language, Phi includes the If expression for such a purpose.

Arguments

  • Predicate : a predicate expression uses to determine if the Then or Else branches should be used.
  • *Then : an expression to be excecuted if the Predicate yields True, since this parameter is variadic you can stack expression and they will be interpreted as a tuple phi.dsl.Seq.

This class also includes the Elif and Else methods which let you write branched conditionals in sequence, however the following rules apply

  • If no branch is entered the whole expression behaves like the identity
  • Elif can only be used after an If or another Elif expression
  • Many Elif expressions can be stacked sequentially
  • Else can only be used after an If or Elif expression

Examples

from phi import P, If

assert "Between 2 and 10" == P.Pipe(
    5,
    If(P > 10,
        "Greater than 10"
    ).Elif(P < 2,
        "Less than 2"
    ).Else(
        "Between 2 and 10"
    )
)
def If(self, condition, *then, **kwargs):
    """
**
If(Predicate, *Then)
ng conditionals expressions a necesity in every language, Phi includes the `If` expression for such a purpose.
guments**
Predicate** : a predicate expression uses to determine if the `Then` or `Else` branches should be used.
*Then** : an expression to be excecuted if the `Predicate` yields `True`, since this parameter is variadic you can stack expression and they will be interpreted as a tuple `phi.dsl.Seq`.
 class also includes the `Elif` and `Else` methods which let you write branched conditionals in sequence, however the following rules apply
 no branch is entered the whole expression behaves like the identity
lif` can only be used after an `If` or another `Elif` expression
ny `Elif` expressions can be stacked sequentially
lse` can only be used after an `If` or `Elif` expression
xamples **
from phi import P, If
assert "Between 2 and 10" == P.Pipe(
    5,
    If(P > 10,
        "Greater than 10"
    ).Elif(P < 2,
        "Less than 2"
    ).Else(
        "Between 2 and 10"
    )
)
    """
    cond_f = _parse(condition)._f
    then_f = E.Seq(*then)._f
    else_f = utils.state_identity
    ast = (cond_f, then_f, else_f)
    g = _compile_if(ast)
    expr = self.__then__(g, **kwargs)
    expr._ast = ast
    expr._root = self
    return expr

def List(

self, *branches, **kwargs)

While Seq is sequential, phi.dsl.Expression.List allows you to split the computation and get back a list with the result of each path. While the list literal should be the most incarnation of this expresion, it can actually be any iterable (implements __iter__) that is not a tuple and yields a valid expresion.

The expression

k = List(f, g)

is equivalent to

k = lambda x: [ f(x), g(x) ]

In general, the following rules apply after compilation:

General Branching

List(f0, f1, ..., fn)

is equivalent to

lambda x: [ f0(x), f1(x), ..., fn(x) ]

Composing & Branching

It is interesting to see how braching interacts with composing. The expression

Seq(f, List(g, h))

is almost equivalent to

List( Seq(f, g), Seq(f, h) )

As you see its as if f where distributed over the List. We say almost because their implementation is different

def _lambda(x):
    x = f(x)
    return [ g(x), h(x) ]

vs

lambda x: [ g(f(x)), h(f(x)) ]

As you see f is only executed once in the first one. Both should yield the same result if f is a pure function.

Examples

form phi import P, List

avg_word_length = P.Pipe(
    "1 22 333",
    lambda s: s.split(' '), # ['1', '22', '333']
    lambda l: map(len, l), # [1, 2, 3]
    List(
        sum # 1 + 2 + 3 == 6
    ,
        len # len([1, 2, 3]) == 3
    ),
    lambda l: l[0] / l[1] # sum / len == 6 / 3 == 2
)

assert avg_word_length == 2

The previous could also be done more briefly like this

form phi import P, Obj, List

avg_word_length = P.Pipe(
    "1 22 333", Obj
    .split(' ')  # ['1', '22', '333']
    .map(len)    # [1, 2, 3]
    .List(
        sum  #sum([1, 2, 3]) == 6
    ,
        len  #len([1, 2, 3]) == 3
    ),
    P[0] / P[1]  #6 / 3 == 2
)

assert avg_word_length == 2

In the example above the last expression

P[0] / P[1]

works for a couple of reasons

  1. The previous expression returns a list
  2. In general the expression P[x] compiles to a function with the form lambda obj: obj[x]
  3. The class Expression (the class from which the object P inherits) overrides most operators to create functions easily. For example, the expression

    (P * 2) / (P + 1)

compile to a function of the form

lambda x: (x * 2) / (x + 1)

Check out the documentatio for Phi lambdas.

def List(self, *branches, **kwargs):
    """
e `Seq` is sequential, `phi.dsl.Expression.List` allows you to split the computation and get back a list with the result of each path. While the list literal should be the most incarnation of this expresion, it can actually be any iterable (implements `__iter__`) that is not a tuple and yields a valid expresion.
expression
k = List(f, g)
quivalent to
k = lambda x: [ f(x), g(x) ]
eneral, the following rules apply after compilation:
neral Branching**
List(f0, f1, ..., fn)
quivalent to
lambda x: [ f0(x), f1(x), ..., fn(x) ]
mposing & Branching**
s interesting to see how braching interacts with composing. The expression
Seq(f, List(g, h))
almost* equivalent to
List( Seq(f, g), Seq(f, h) )
ou see its as if `f` where distributed over the List. We say *almost* because their implementation is different
def _lambda(x):
    x = f(x)
    return [ g(x), h(x) ]
lambda x: [ g(f(x)), h(f(x)) ]
ou see `f` is only executed once in the first one. Both should yield the same result if `f` is a pure function.
Examples
form phi import P, List
avg_word_length = P.Pipe(
    "1 22 333",
    lambda s: s.split(' '), # ['1', '22', '333']
    lambda l: map(len, l), # [1, 2, 3]
    List(
        sum # 1 + 2 + 3 == 6
    ,
        len # len([1, 2, 3]) == 3
    ),
    lambda l: l[0] / l[1] # sum / len == 6 / 3 == 2
)
assert avg_word_length == 2
previous could also be done more briefly like this
form phi import P, Obj, List
avg_word_length = P.Pipe(
    "1 22 333", Obj
    .split(' ')  # ['1', '22', '333']
    .map(len)    # [1, 2, 3]
    .List(
        sum  #sum([1, 2, 3]) == 6
    ,
        len  #len([1, 2, 3]) == 3
    ),
    P[0] / P[1]  #6 / 3 == 2
)
assert avg_word_length == 2
he example above the last expression
P[0] / P[1]
s for a couple of reasons
he previous expression returns a list
n general the expression `P[x]` compiles to a function with the form `lambda obj: obj[x]`
he class `Expression` (the class from which the object `P` inherits) overrides most operators to create functions easily. For example, the expression
(P * 2) / (P + 1)
ile to a function of the form
lambda x: (x * 2) / (x + 1)
k out the documentatio for Phi [lambdas](https://cgarciae.github.io/phi/lambdas.m.html).
    """
    gs = [ _parse(code)._f for code in branches ]
    def h(x, state):
        ys = []
        for g in gs:
            y, state = g(x, state)
            ys.append(y)
        return (ys, state)
    return self.__then__(h, **kwargs)

def PatchAt(

cls, n, module, method_wrapper=None, module_alias=None, method_name_modifier=<function identity at 0x7fea7e92ea28>, blacklist_predicate=<function <lambda> at 0x7fea7e8e27d0>, whitelist_predicate=<function <lambda> at 0x7fea7e8e2758>, return_type_predicate=<function <lambda> at 0x7fea7e8e2848>, getmembers_predicate=<function isfunction at 0x7fea7fdff488>, admit_private=False, explanation=u'')

This classmethod lets you easily patch all of functions/callables from a module or class as methods a Builder class.

Arguments

  • n : the position the the object being piped will take in the arguments when the function being patched is applied. See RegisterMethod and ThenAt.
  • module : a module or class from which the functions/methods/callables will be taken.
  • module_alias = None : an optional alias for the module used for documentation purposes.
  • method_name_modifier = lambda f_name: None : a function that can modify the name of the method will take. If None the name of the function will be used.
  • blacklist_predicate = lambda f_name: name[0] != "_" : A predicate that determines which functions are banned given their name. By default it excludes all function whose name start with '_'. blacklist_predicate can also be of type list, in which case all names contained in this list will be banned.
  • whitelist_predicate = lambda f_name: True : A predicate that determines which functions are admitted given their name. By default it include any function. whitelist_predicate can also be of type list, in which case only names contained in this list will be admitted. You can use both blacklist_predicate and whitelist_predicate at the same time.
  • return_type_predicate = lambda f_name: None : a predicate that determines the _return_type of the Builder. By default it will always return None. See ThenAt.
  • getmembers_predicate = inspect.isfunction : a predicate that determines what type of elements/members will be fetched by the inspect module, defaults to inspect.isfunction. See getmembers.

Examples

Lets patch ALL the main functions from numpy into a custom builder!

from phi import PythonBuilder #or Builder
import numpy as np

class NumpyBuilder(PythonBuilder): #or Builder
    "A Builder for numpy functions!"
    pass

NumpyBuilder.PatchAt(1, np)

N = NumpyBuilder(lambda x: x)

Thats it! Although a serious patch would involve filtering out functions that don't take arrays. Another common task would be to use NumpyBuilder.PatchAt(2, ...) (PatchAt(n, ..) in general) when convenient to send the object being pipe to the relevant argument of the function. The previous is usually done with and a combination of whitelist_predicates and blacklist_predicates on PatchAt(1, ...) and PatchAt(2, ...) to filter or include the approriate functions on each kind of patch. Given the previous code we could now do

import numpy as np

x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])

z = N.Pipe(
    x, N
    .dot(y)
    .add(x)
    .transpose()
    .sum(axis=1)
)

Which is strictly equivalent to

import numpy as np

x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])

z = np.dot(x, y)
z = np.add(z, x)
z = np.transpose(z)
z = np.sum(z, axis=1)

The thing to notice is that with the NumpyBuilder we avoid the repetitive and needless passing and reassigment of the z variable, this removes a lot of noise from our code.

@classmethod
def PatchAt(cls, n, module, method_wrapper=None, module_alias=None, method_name_modifier=utils.identity, blacklist_predicate=_False, whitelist_predicate=_True, return_type_predicate=_None, getmembers_predicate=inspect.isfunction, admit_private=False, explanation=""):
    """
 classmethod lets you easily patch all of functions/callables from a module or class as methods a Builder class.
guments**
n** : the position the the object being piped will take in the arguments when the function being patched is applied. See `RegisterMethod` and `ThenAt`.
module** : a module or class from which the functions/methods/callables will be taken.
odule_alias = None` : an optional alias for the module used for documentation purposes.
ethod_name_modifier = lambda f_name: None` : a function that can modify the name of the method will take. If `None` the name of the function will be used.
lacklist_predicate = lambda f_name: name[0] != "_"` : A predicate that determines which functions are banned given their name. By default it excludes all function whose name start with `'_'`. `blacklist_predicate` can also be of type list, in which case all names contained in this list will be banned.
hitelist_predicate = lambda f_name: True` : A predicate that determines which functions are admitted given their name. By default it include any function. `whitelist_predicate` can also be of type list, in which case only names contained in this list will be admitted. You can use both `blacklist_predicate` and `whitelist_predicate` at the same time.
eturn_type_predicate = lambda f_name: None` : a predicate that determines the `_return_type` of the Builder. By default it will always return `None`. See `phi.builder.Builder.ThenAt`.
etmembers_predicate = inspect.isfunction` : a predicate that determines what type of elements/members will be fetched by the `inspect` module, defaults to [inspect.isfunction](https://docs.python.org/2/library/inspect.html#inspect.isfunction). See [getmembers](https://docs.python.org/2/library/inspect.html#inspect.getmembers).
amples**
 patch ALL the main functions from numpy into a custom builder!
from phi import PythonBuilder #or Builder
import numpy as np
class NumpyBuilder(PythonBuilder): #or Builder
    "A Builder for numpy functions!"
    pass
NumpyBuilder.PatchAt(1, np)
N = NumpyBuilder(lambda x: x)
s it! Although a serious patch would involve filtering out functions that don't take arrays. Another common task would be to use `NumpyBuilder.PatchAt(2, ...)` (`PatchAt(n, ..)` in general) when convenient to send the object being pipe to the relevant argument of the function. The previous is usually done with and a combination of `whitelist_predicate`s and `blacklist_predicate`s on `PatchAt(1, ...)` and `PatchAt(2, ...)` to filter or include the approriate functions on each kind of patch. Given the previous code we could now do
import numpy as np
x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
z = N.Pipe(
    x, N
    .dot(y)
    .add(x)
    .transpose()
    .sum(axis=1)
)
h is strictly equivalent to
import numpy as np
x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
z = np.dot(x, y)
z = np.add(z, x)
z = np.transpose(z)
z = np.sum(z, axis=1)
thing to notice is that with the `NumpyBuilder` we avoid the repetitive and needless passing and reassigment of the `z` variable, this removes a lot of noise from our code.
    """
    _rtp = return_type_predicate
    return_type_predicate = (lambda x: _rtp) if inspect.isclass(_rtp) and issubclass(_rtp, Builder) else _rtp
    module_name = module_alias if module_alias else module.__name__ + '.'
    patch_members = _get_patch_members(module, blacklist_predicate=blacklist_predicate, whitelist_predicate=whitelist_predicate, getmembers_predicate=getmembers_predicate, admit_private=admit_private)
    if type(whitelist_predicate) is list and "convolution2d" in whitelist_predicate:
        import ipdb
        #ipdb.set_trace()
    for name, f in patch_members:
        wrapped = None
        if method_wrapper:
            g = method_wrapper(f)
            wrapped = f
        else:
            g = f
        cls.RegisterAt(n, g, module_name, wrapped=wrapped, _return_type=return_type_predicate(name), alias=method_name_modifier(name), explanation=explanation)

def Pipe(

self, *sequence, **kwargs)

Pipe runs any phi.dsl.Expression. Its highly inspired by Elixir's |> (pipe) operator.

Arguments

  • *sequence: any variable amount of expressions. All expressions inside of sequence will be composed together using phi.dsl.Expression.Seq.
  • **kwargs: Pipe forwards all kwargs to Seq, visit its documentation for more info.

The expression

Pipe(*sequence, **kwargs)

is equivalent to

Seq(*sequence, **kwargs)(None)

Normally the first argument or Pipe is a value, that is reinterpreted as a phi.dsl.Expression.Val, therfore, the input None is discarded.

Examples

from phi import P

def add1(x): return x + 1
def mul3(x): return x * 3

x = P.Pipe(
    1,     #input
    add1,  #1 + 1 == 2
    mul3   #2 * 3 == 6
)

assert x == 6

The previous using lambdas to create the functions

from phi import P

x = P.Pipe(
    1,      #input
    P + 1,  #1 + 1 == 2
    P * 3   #2 * 3 == 6
)

assert x == 6

Also see

def Pipe(self, *sequence, **kwargs):
    """
e` runs any `phi.dsl.Expression`. Its highly inspired by Elixir's [|> (pipe)](https://hexdocs.pm/elixir/Kernel.html#%7C%3E/2) operator.
guments**
*sequence**: any variable amount of expressions. All expressions inside of `sequence` will be composed together using `phi.dsl.Expression.Seq`.
**kwargs**: `Pipe` forwards all `kwargs` to `phi.builder.Builder.Seq`, visit its documentation for more info.
expression
Pipe(*sequence, **kwargs)
quivalent to
Seq(*sequence, **kwargs)(None)
ally the first argument or `Pipe` is a value, that is reinterpreted as a `phi.dsl.Expression.Val`, therfore, the input `None` is discarded.
amples**
from phi import P
def add1(x): return x + 1
def mul3(x): return x * 3
x = P.Pipe(
    1,     #input
    add1,  #1 + 1 == 2
    mul3   #2 * 3 == 6
)
assert x == 6
previous using [lambdas](https://cgarciae.github.io/phi/lambdas.m.html) to create the functions
from phi import P
x = P.Pipe(
    1,      #input
    P + 1,  #1 + 1 == 2
    P * 3   #2 * 3 == 6
)
assert x == 6
so see**
hi.builder.Builder.Seq`
sl](https://cgarciae.github.io/phi/dsl.m.html)
ompile](https://cgarciae.github.io/phi/dsl.m.html#phi.dsl.Compile)
ambdas](https://cgarciae.github.io/phi/lambdas.m.html)
    """
    state = kwargs.pop("refs", {})
    return self.Seq(*sequence, **kwargs)(None, **state)

def ReadList(

self, *branches, **kwargs)

Same as phi.dsl.Expression.List but any string argument x is translated to Read(x).

def ReadList(self, *branches, **kwargs):
    """
 as `phi.dsl.Expression.List` but any string argument `x` is translated to `Read(x)`.
    """
    branches = map(lambda x: E.Read(x) if isinstance(x, str) else x, branches)
    return self.List(*branches, **kwargs)

def Register(

cls, *args, **kwargs)

Register(...) is a shortcut for RegisterAt(1, ...)

Also See

@classmethod
def Register(cls, *args, **kwargs):
    """
ister(...)` is a shortcut for `RegisterAt(1, ...)`
so See**
hi.builder.Builder.RegisterAt`
hi.builder.Builder.RegisterMethod`
    """
    return cls.RegisterAt(1, *args, **kwargs)

def Register0(

cls, *args, **kwargs)

Register0(...) is a shortcut for RegisterAt(0, ...)

Also See

@classmethod
def Register0(cls, *args, **kwargs):
    """
ister0(...)` is a shortcut for `RegisterAt(0, ...)`
so See**
hi.builder.Builder.RegisterAt`
hi.builder.Builder.RegisterMethod`
    """
    return cls.RegisterAt(0, *args, **kwargs)

def Register2(

cls, *args, **kwargs)

Register2(...) is a shortcut for RegisterAt(2, ...)

Also See

@classmethod
def Register2(cls, *args, **kwargs):
    """
ister2(...)` is a shortcut for `RegisterAt(2, ...)`
so See**
hi.builder.Builder.RegisterAt`
hi.builder.Builder.RegisterMethod`
    """
    return cls.RegisterAt(2, *args, **kwargs)

def Register3(

cls, *args, **kwargs)

Register3(...) is a shortcut for RegisterAt(3, ...)

Also See

@classmethod
def Register3(cls, *args, **kwargs):
    """
ister3(...)` is a shortcut for `RegisterAt(3, ...)`
so See**
hi.builder.Builder.RegisterAt`
hi.builder.Builder.RegisterMethod`
    """
    return cls.RegisterAt(3, *args, **kwargs)

def Register4(

cls, *args, **kwargs)

Register4(...) is a shortcut for RegisterAt(4, ...)

Also See

@classmethod
def Register4(cls, *args, **kwargs):
    """
ister4(...)` is a shortcut for `RegisterAt(4, ...)`
so See**
hi.builder.Builder.RegisterAt`
hi.builder.Builder.RegisterMethod`
    """
    return cls.RegisterAt(4, *args, **kwargs)

def Register5(

cls, *args, **kwargs)

Register5(...) is a shortcut for RegisterAt(5, ...)

Also See

@classmethod
def Register5(cls, *args, **kwargs):
    """
ister5(...)` is a shortcut for `RegisterAt(5, ...)`
so See**
hi.builder.Builder.RegisterAt`
hi.builder.Builder.RegisterMethod`
    """
    return cls.RegisterAt(5, *args, **kwargs)

def RegisterAt(

cls, *args, **kwargs)

RegisterAt

RegisterAt(n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None)

Most of the time you don't want to register an method as such, that is, you don't care about the self builder object, instead you want to register a function that transforms the value being piped down the DSL. For this you can use RegisterAt so e.g.

def some_fun(obj, arg1, arg2):
    # code

@MyBuilder.RegisterMethod("my_lib.")
def some_fun_wrapper(self, arg1, arg2):
    return self.ThenAt(1, some_fun, arg1, arg2)

can be written directly as

@MyBuilder.RegisterAt(1, "my_lib.")
def some_fun(obj, arg1, arg2):
    # code

For this case you can just use Register which is a shortcut for RegisterAt(1, ...)

@MyBuilder.Register("my_lib.")
def some_fun(obj, arg1, arg2):
    # code

Also See

@classmethod
def RegisterAt(cls, *args, **kwargs):
    """
gisterAt**
RegisterAt(n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None)
 of the time you don't want to register an method as such, that is, you don't care about the `self` builder object, instead you want to register a function that transforms the value being piped down the DSL. For this you can use `RegisterAt` so e.g.
def some_fun(obj, arg1, arg2):
    # code
@MyBuilder.RegisterMethod("my_lib.")
def some_fun_wrapper(self, arg1, arg2):
    return self.ThenAt(1, some_fun, arg1, arg2)
be written directly as
@MyBuilder.RegisterAt(1, "my_lib.")
def some_fun(obj, arg1, arg2):
    # code
this case you can just use `Register` which is a shortcut for `RegisterAt(1, ...)`
@MyBuilder.Register("my_lib.")
def some_fun(obj, arg1, arg2):
    # code
so See**
hi.builder.Builder.RegisterMethod`
    """
    unpack_error = True
    try:
        n, f, library_path = args
        unpack_error = False
        cls._RegisterAt(n, f, library_path, **kwargs)
    except:
        if not unpack_error:
            raise
        def register_decorator(f):
            n, library_path = args
            cls._RegisterAt(n, f, library_path, **kwargs)
            return f
        return register_decorator

def RegisterMethod(

cls, *args, **kwargs)

RegisterMethod

RegisterMethod(f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True)

classmethod for registering functions as methods of this class.

Arguments

  • f : the particular function being registered as a method
  • library_path : library from where f comes from, unless you pass an empty string, put a period "." at the end of the library name.
  • alias=None : alias for the name/method being registered
  • original_name=None : name of the original function, used for documentation purposes.
  • doc=None : complete documentation of the method being registered
  • wrapped=None : if you are registering a function which wraps around another function, pass this other function through wrapped to get better documentation, this is specially useful is you register a bunch of functions in a for loop. Please include an explanation to tell how the actual function differs from the wrapped one.
  • explanation="" : especify any additional information for the documentation of the method being registered, you can use any of the following format tags within this string and they will be replace latter on: {original_name}, {name}, {fn_docs}, {library_path}, {builder_class}.
  • method_type=identity : by default its applied but does nothing, you might also want to register functions as property, classmethod, staticmethod
  • explain=True : decide whether or not to show any kind of explanation, its useful to set it to False if you are using a Register* decorator and will only use the function as a registered method.

A main feature of phi is that it enables you to integrate your library or even an existing library with the DSL. You can achieve three levels of integration

  1. Passing your functions to the DSL. This a very general machanism -since you could actually do everything with python lamdas- but in practice functions often receive multiple parameters.
  2. Creating partials with the Then* method family. Using this you could integrate any function, but it will add a lot of noise if you use heavily on it.
  3. Registering functions as methods of a Builder derived class. This produces the most readable code and its the approach you should take if you want to create a Phi-based library or a helper class.

While point 3 is the most desirable it has a cost: you need to create your own Builder-derived class. This is because SHOULD NOT register functions to existing builders e.g. the Builder or PythonBuilder provided by phi because that would pollute the P object. Instead you should create a custom class that derives from Builder, PythonBuilder or another custom builder depending on your needs and register your functions to that class.

Examples

Say you have a function on a library called "my_lib"

def some_fun(obj, arg1, arg2):
    # code

You could use it with the dsl like this

from phi import P, Then

P.Pipe(
    input,
    ...
    Then(some_fun, arg1, arg2)
    ...
)

assuming the first parameter obj is being piped down. However if you do this very often or you are creating a library, you are better off creating a custom class derived from Builder or PythonBuilder

from phi import Builder #or PythonBuilder

class MyBuilder(Builder): # or PythonBuilder
    pass

and registering your function as a method. The first way you could do this is by creating a wrapper function for some_fun and registering it as a method

def some_fun_wrapper(self, arg1, arg2):
    return self.Then(some_fun, arg1, arg2)

MyBuilder.RegisterMethod(some_fun_wrapper, "my_lib.", wrapped=some_fun)

Here we basically created a shortcut for the original expression Then(some_fun, arg1, arg2). You could also do this using a decorator

@MyBuilder.RegisterMethod("my_lib.", wrapped=some_fun)
def some_fun_wrapper(self, arg1, arg2):
    return self.Then(some_fun, arg1, arg2)

However, this is such a common task that we've created the method Register to avoid you from having to create the wrapper. With it you could register the function some_fun directly as a method like this

MyBuilder.Register(some_fun, "my_lib.")

or by using a decorator over the original function definition

@MyBuilder.Register("my_lib.")
def some_fun(obj, arg1, arg2):
    # code

Once done you've done any of the previous approaches you can create a custom global object e.g. M and use it instead of/along with P

M = MyBuilder(lambda x: x)

M.Pipe(
    input,
    ...
    M.some_fun(arg1, args)
    ...
)

Argument position

Register internally uses Then, this is only useful if the object being piped is intended to be passed as the first argument of the function being registered, if this is not the case you could use Register2, Register3, ..., Register5 or RegisterAt to set an arbitrary position, these functions will internally use Then2, Then3, ..., Then5 or ThenAt respectively.

Wrapping functions

Sometimes you have an existing function that you would like to modify slightly so it plays nicely with the DSL, what you normally do is create a function that wraps around it and passes the arguments to it in a way that is convenient

import some_lib

@MyBuilder.Register("some_lib.")
def some_fun(a, n):
    return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified

When you do this -as a side effect- you loose the original documentation, to avoid this you can use the Registers wrapped argument along with the explanation argument to clarity the situation

import some_lib

some_fun_explanation = "However, it differs in that `n` is automatically subtracted `1`"

@MyBuilder.Register("some_lib.", wrapped=some_lib.some_fun, explanation=some_fun_explanation)
def some_fun(a, n):
    return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified

Now the documentation for MyBuilder.some_fun will be a little bit nicer since it includes the original documentation from some_lib.some_fun. This behaviour is specially useful if you are wrapping an entire 3rd party library, you usually automate the process iterating over all the funcitions in a for loop. The PatchAt method lets you register and entire module using a few lines of code, however, something you have to do thing more manually and do the iteration yourself.

See Also

@classmethod
def RegisterMethod(cls, *args, **kwargs):
    """
gisterMethod**
RegisterMethod(f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True)
ssmethod` for registering functions as methods of this class.
guments**
f** : the particular function being registered as a method
library_path** : library from where `f` comes from, unless you pass an empty string, put a period `"."` at the end of the library name.
lias=None` : alias for the name/method being registered
riginal_name=None` : name of the original function, used for documentation purposes.
oc=None` : complete documentation of the method being registered
rapped=None` : if you are registering a function which wraps around another function, pass this other function through `wrapped` to get better documentation, this is specially useful is you register a bunch of functions in a for loop. Please include an `explanation` to tell how the actual function differs from the wrapped one.
xplanation=""` : especify any additional information for the documentation of the method being registered, you can use any of the following format tags within this string and they will be replace latter on: `{original_name}`, `{name}`, `{fn_docs}`, `{library_path}`, `{builder_class}`.
ethod_type=identity` : by default its applied but does nothing, you might also want to register functions as `property`, `classmethod`, `staticmethod`
xplain=True` : decide whether or not to show any kind of explanation, its useful to set it to `False` if you are using a `Register*` decorator and will only use the function as a registered method.
in feature of `phi` is that it enables you to integrate your library or even an existing library with the DSL. You can achieve three levels of integration
assing your functions to the DSL. This a very general machanism -since you could actually do everything with python lamdas- but in practice functions often receive multiple parameters.
reating partials with the `Then*` method family. Using this you could integrate any function, but it will add a lot of noise if you use heavily on it.
egistering functions as methods of a `Builder` derived class. This produces the most readable code and its the approach you should take if you want to create a Phi-based library or a helper class.
e point 3 is the most desirable it has a cost: you need to create your own `phi.builder.Builder`-derived class. This is because SHOULD NOT register functions to existing builders e.g. the `phi.builder.Builder` or [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) provided by phi because that would pollute the `P` object. Instead you should create a custom class that derives from `phi.builder.Builder`,  [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) or another custom builder depending on your needs and register your functions to that class.
amples**
you have a function on a library called `"my_lib"`
def some_fun(obj, arg1, arg2):
    # code
could use it with the dsl like this
from phi import P, Then
P.Pipe(
    input,
    ...
    Then(some_fun, arg1, arg2)
    ...
)
ming the first parameter `obj` is being piped down. However if you do this very often or you are creating a library, you are better off creating a custom class derived from `Builder` or `PythonBuilder`
from phi import Builder #or PythonBuilder
class MyBuilder(Builder): # or PythonBuilder
    pass
registering your function as a method. The first way you could do this is by creating a wrapper function for `some_fun` and registering it as a method
def some_fun_wrapper(self, arg1, arg2):
    return self.Then(some_fun, arg1, arg2)
MyBuilder.RegisterMethod(some_fun_wrapper, "my_lib.", wrapped=some_fun)
 we basically created a shortcut for the original expression `Then(some_fun, arg1, arg2)`. You could also do this using a decorator
@MyBuilder.RegisterMethod("my_lib.", wrapped=some_fun)
def some_fun_wrapper(self, arg1, arg2):
    return self.Then(some_fun, arg1, arg2)
ver, this is such a common task that we've created the method `Register` to avoid you from having to create the wrapper. With it you could register the function `some_fun` directly as a method like this
MyBuilder.Register(some_fun, "my_lib.")
y using a decorator over the original function definition
@MyBuilder.Register("my_lib.")
def some_fun(obj, arg1, arg2):
    # code
 done you've done any of the previous approaches you can create a custom global object e.g. `M` and use it instead of/along with `P`
M = MyBuilder(lambda x: x)
M.Pipe(
    input,
    ...
    M.some_fun(arg1, args)
    ...
)
gument position**
.builder.Builder.Register` internally uses `phi.builder.Builder.Then`, this is only useful if the object being piped is intended to be passed as the first argument of the function being registered, if this is not the case you could use `phi.builder.Builder.Register2`, `phi.builder.Builder.Register3`, ..., `phi.builder.Builder.Register5` or `phi.builder.Builder.RegisterAt` to set an arbitrary position, these functions will internally use `phi.builder.Builder.Then2`, `phi.builder.Builder.Then3`, ..., `phi.builder.Builder.Then5` or `phi.builder.Builder.ThenAt` respectively.
apping functions**
times you have an existing function that you would like to modify slightly so it plays nicely with the DSL, what you normally do is create a function that wraps around it and passes the arguments to it in a way that is convenient
import some_lib
@MyBuilder.Register("some_lib.")
def some_fun(a, n):
    return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified
 you do this -as a side effect- you loose the original documentation, to avoid this you can use the Registers `wrapped` argument along with the `explanation` argument to clarity the situation
import some_lib
some_fun_explanation = "However, it differs in that `n` is automatically subtracted `1`"
@MyBuilder.Register("some_lib.", wrapped=some_lib.some_fun, explanation=some_fun_explanation)
def some_fun(a, n):
    return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified
the documentation for `MyBuilder.some_fun` will be a little bit nicer since it includes the original documentation from `some_lib.some_fun`. This behaviour is specially useful if you are wrapping an entire 3rd party library, you usually automate the process iterating over all the funcitions in a for loop. The `phi.builder.Builder.PatchAt` method lets you register and entire module using a few lines of code, however, something you have to do thing more manually and do the iteration yourself.
e Also**
hi.builder.Builder.PatchAt`
hi.builder.Builder.RegisterAt`
    """
    unpack_error = True
    try:
        f, library_path = args
        unpack_error = False
        cls._RegisterMethod(f, library_path, **kwargs)
    except:
        if not unpack_error:
            raise
        def register_decorator(f):
            library_path, = args
            cls._RegisterMethod(f, library_path, **kwargs)
            return f
        return register_decorator

def Seq(

self, *sequence, **kwargs)

Seq is used to express function composition. The expression

Seq(f, g)

be equivalent to

lambda x: g(f(x))

As you see, its a little different from the mathematical definition. Excecution order flow from left to right, this makes reading and reasoning about code way more easy. This bahaviour is based upon the |> (pipe) operator found in languages like F#, Elixir and Elm. You can pack as many expressions as you like and they will be applied in order to the data that is passed through them when compiled an excecuted.

In general, the following rules apply for Seq:

General Sequence

Seq(f0, f1, ..., fn-1, fn)

is equivalent to

lambda x: fn(fn-1(...(f1(f0(x)))))

Single Function

Seq(f)

is equivalent to

f

Identity

The empty Seq

Seq()

is equivalent to

lambda x: x

Examples

from phi import P, Seq

f = Seq(
    P * 2,
    P + 1,
    P ** 2
)

assert f(1) == 9 # ((1 * 2) + 1) ** 2

The previous example using P.Pipe

from phi import P

assert 9 == P.Pipe(
    1,
    P * 2,  #1 * 2 == 2
    P + 1,  #2 + 1 == 3
    P ** 2  #3 ** 2 == 9
)
def Seq(self, *sequence, **kwargs):
    """
` is used to express function composition. The expression
Seq(f, g)
quivalent to
lambda x: g(f(x))
ou see, its a little different from the mathematical definition. Excecution order flow from left to right, this makes reading and reasoning about code way more easy. This bahaviour is based upon the `|>` (pipe) operator found in languages like F#, Elixir and Elm. You can pack as many expressions as you like and they will be applied in order to the data that is passed through them when compiled an excecuted.
eneral, the following rules apply for Seq:
neral Sequence**
Seq(f0, f1, ..., fn-1, fn)
quivalent to
lambda x: fn(fn-1(...(f1(f0(x)))))
ngle Function**
Seq(f)
quivalent to
f
entity**
empty Seq
Seq()
quivalent to
lambda x: x
Examples
from phi import P, Seq
f = Seq(
    P * 2,
    P + 1,
    P ** 2
)
assert f(1) == 9 # ((1 * 2) + 1) ** 2
previous example using `P.Pipe`
from phi import P
assert 9 == P.Pipe(
    1,
    P * 2,  #1 * 2 == 2
    P + 1,  #2 + 1 == 3
    P ** 2  #3 ** 2 == 9
)
    """
    fs = [ _parse(elem)._f for elem in sequence ]
    def g(x, state):
        return functools.reduce(lambda args, f: f(*args), fs, (x, state))
    return self.__then__(g, **kwargs)

def Set(

self, *expressions, **kwargs)

def Set(self, *expressions, **kwargs):
    return self.List(*expressions) >> set

def Then(

self, f, *args, **kwargs)

Then(f, ...) is equivalent to ThenAt(1, f, ...). Checkout ThenAt for more information.

def Then(self, f, *args, **kwargs):
    """
n(f, ...)` is equivalent to `ThenAt(1, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information.
    """
    return self.ThenAt(1, f, *args, **kwargs)

def Then0(

self, f, *args, **kwargs)

Then0(f, ...) is equivalent to ThenAt(0, f, ...). Checkout ThenAt for more information.

def Then0(self, f, *args, **kwargs):
    """
n0(f, ...)` is equivalent to `ThenAt(0, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information.
    """
    return self.ThenAt(0, f, *args, **kwargs)

def Then1(

self, f, *args, **kwargs)

Then(f, ...) is equivalent to ThenAt(1, f, ...). Checkout ThenAt for more information.

def Then(self, f, *args, **kwargs):
    """
n(f, ...)` is equivalent to `ThenAt(1, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information.
    """
    return self.ThenAt(1, f, *args, **kwargs)

def Then2(

self, f, arg1, *args, **kwargs)

Then2(f, ...) is equivalent to ThenAt(2, f, ...). Checkout ThenAt for more information.

def Then2(self, f, arg1, *args, **kwargs):
    """
n2(f, ...)` is equivalent to `ThenAt(2, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information.
    """
    args = (arg1,) + args
    return self.ThenAt(2, f, *args, **kwargs)

def Then3(

self, f, arg1, arg2, *args, **kwargs)

Then3(f, ...) is equivalent to ThenAt(3, f, ...). Checkout ThenAt for more information.

def Then3(self, f, arg1, arg2, *args, **kwargs):
    """
n3(f, ...)` is equivalent to `ThenAt(3, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information.
    """
    args = (arg1, arg2) + args
    return self.ThenAt(3, f, *args, **kwargs)

def Then4(

self, f, arg1, arg2, arg3, *args, **kwargs)

Then4(f, ...) is equivalent to ThenAt(4, f, ...). Checkout ThenAt for more information.

def Then4(self, f, arg1, arg2, arg3, *args, **kwargs):
    """
n4(f, ...)` is equivalent to `ThenAt(4, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information.
    """
    args = (arg1, arg2, arg3) + args
    return self.ThenAt(4, f, *args, **kwargs)

def Then5(

self, f, arg1, arg2, arg3, arg4, *args, **kwargs)

Then5(f, ...) is equivalent to ThenAt(5, f, ...). Checkout ThenAt for more information.

def Then5(self, f, arg1, arg2, arg3, arg4, *args, **kwargs):
    """
n5(f, ...)` is equivalent to `ThenAt(5, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information.
    """
    args = (arg1, arg2, arg3, arg4) + args
    return self.ThenAt(5, f, *args, **kwargs)

def ThenAt(

self, n, f, *_args, **kwargs)

ThenAt enables you to create a partially apply many arguments to a function, the returned partial expects a single arguments which will be applied at the nth position of the original function.

Arguments

  • n: position at which the created partial will apply its awaited argument on the original function.
  • f: function which the partial will be created.
  • _args & kwargs: all *_args and **kwargs will be passed to the function f.
  • _return_type = None: type of the returned builder, if None it will return the same type of the current builder. This special kwarg will NOT be passed to f.

You can think of n as the position that the value being piped down will pass through the f. Say you have the following expression

D == fun(A, B, C)

all the following are equivalent

from phi import P, Pipe, ThenAt

D == Pipe(A, ThenAt(1, fun, B, C))
D == Pipe(B, ThenAt(2, fun, A, C))
D == Pipe(C, ThenAt(3, fun, A, B))

you could also use the shortcuts Then, Then2,..., Then5, which are more readable

from phi import P, Pipe

D == Pipe(A, P.Then(fun, B, C))
D == Pipe(B, P.Then2(fun, A, C))
D == Pipe(C, P.Then3(fun, A, B))

There is a special case not discussed above: n = 0. When this happens only the arguments given will be applied to f, this method it will return a partial that expects a single argument but completely ignores it

from phi import P

D == Pipe(None, P.ThenAt(0, fun, A, B, C))
D == Pipe(None, P.Then0(fun, A, B, C))

Examples

Max of 6 and the argument:

from phi import P

assert 6 == P.Pipe(
    2,
    P.Then(max, 6)
)

Previous is equivalent to

assert 6 == max(2, 6)

Open a file in read mode ('r')

from phi import P

f = P.Pipe(
    "file.txt",
    P.Then(open, 'r')
)

Previous is equivalent to

f = open("file.txt", 'r')

Split a string by whitespace and then get the length of each word

from phi import P

assert [5, 5, 5] == P.Pipe(
    "Again hello world",
    P.Then(str.split, ' ')
    .Then2(map, len)
)

Previous is equivalent to

x = "Again hello world"

x = str.split(x, ' ')
x = map(len, x)

assert [5, 5, 5] == x

As you see, Then2 was very useful because map accepts and iterable as its 2nd parameter. You can rewrite the previous using the PythonBuilder and the Obj object

from phi import P, Obj

assert [5, 5, 5] == P.Pipe(
    "Again hello world",
    Obj.split(' '),
    P.map(len)
)

Also see

def ThenAt(self, n, f, *_args, **kwargs):
    """
nAt` enables you to create a partially apply many arguments to a function, the returned partial expects a single arguments which will be applied at the `n`th position of the original function.
guments**
n**: position at which the created partial will apply its awaited argument on the original function.
f**: function which the partial will be created.
_args & kwargs**: all `*_args` and `**kwargs` will be passed to the function `f`.
return_type = None`: type of the returned `builder`, if `None` it will return the same type of the current `builder`. This special kwarg will NOT be passed to `f`.
can think of `n` as the position that the value being piped down will pass through the `f`. Say you have the following expression
D == fun(A, B, C)
the following are equivalent
from phi import P, Pipe, ThenAt
D == Pipe(A, ThenAt(1, fun, B, C))
D == Pipe(B, ThenAt(2, fun, A, C))
D == Pipe(C, ThenAt(3, fun, A, B))
could also use the shortcuts `Then`, `Then2`,..., `Then5`, which are more readable
from phi import P, Pipe
D == Pipe(A, P.Then(fun, B, C))
D == Pipe(B, P.Then2(fun, A, C))
D == Pipe(C, P.Then3(fun, A, B))
e is a special case not discussed above: `n = 0`. When this happens only the arguments given will be applied to `f`, this method it will return a partial that expects a single argument but completely ignores it
from phi import P
D == Pipe(None, P.ThenAt(0, fun, A, B, C))
D == Pipe(None, P.Then0(fun, A, B, C))
amples**
of 6 and the argument:
from phi import P
assert 6 == P.Pipe(
    2,
    P.Then(max, 6)
)
ious is equivalent to
assert 6 == max(2, 6)
 a file in read mode (`'r'`)
from phi import P
f = P.Pipe(
    "file.txt",
    P.Then(open, 'r')
)
ious is equivalent to
f = open("file.txt", 'r')
t a string by whitespace and then get the length of each word
from phi import P
assert [5, 5, 5] == P.Pipe(
    "Again hello world",
    P.Then(str.split, ' ')
    .Then2(map, len)
)
ious is equivalent to
x = "Again hello world"
x = str.split(x, ' ')
x = map(len, x)
assert [5, 5, 5] == x
ou see, `Then2` was very useful because `map` accepts and `iterable` as its `2nd` parameter. You can rewrite the previous using the [PythonBuilder](https://cgarciae.github.io/phi/python_builder.m.html) and the `phi.builder.Builder.Obj` object
from phi import P, Obj
assert [5, 5, 5] == P.Pipe(
    "Again hello world",
    Obj.split(' '),
    P.map(len)
)
so see**
hi.builder.Builder.Obj`
ythonBuilder](https://cgarciae.github.io/phi/python_builder.m.html)
hi.builder.Builder.RegisterAt`
    """
    _return_type = None
    n_args = n - 1
    if '_return_type' in kwargs:
        _return_type = kwargs['_return_type']
        del kwargs['_return_type']
    @utils.lift
    def g(x):
        new_args = _args[0:n_args] + (x,) + _args[n_args:] if n_args >= 0 else _args
        return f(*new_args, **kwargs)
    return self.__then__(g, _return_type=_return_type)

def Tuple(

self, *expressions, **kwargs)

def Tuple(self, *expressions, **kwargs):
    return self.List(*expressions) >> tuple

def Val(

self, val, **kwargs)

The expression

Val(a)

is equivalent to the constant function

lambda x: a

All expression in this module interprete values that are not functions as constant functions using Val, for example

Seq(1, P + 1)

is equivalent to

Seq(Val(1), P + 1)

The previous expression as a whole is a constant function since it will return 2 no matter what input you give it.

def Val(self, val, **kwargs):
    """
expression
Val(a)
quivalent to the constant function
lambda x: a
expression in this module interprete values that are not functions as constant functions using `Val`, for example
Seq(1, P + 1)
quivalent to
Seq(Val(1), P + 1)
previous expression as a whole is a constant function since it will return `2` no matter what input you give it.
    """
    f = utils.lift(lambda z: val)
    return self.__then__(f, **kwargs)

def With(

self, context_manager, *body, **kwargs)

With

def With(context_manager, *body):

Arguments

  • context_manager: a context manager object or valid expression from the DSL that returns a context manager.
  • *body: any valid expression of the DSL to be evaluated inside the context. *body is interpreted as a tuple so all expression contained are composed.

As with normal python programs you sometimes might want to create a context for a block of code. You normally give a context manager to the with statemente, in Phi you use P.With or phi.With

Context

Python's with statemente returns a context object through as keyword, in the DSL this object can be obtained using the P.Context method or the phi.Context function.

Examples

from phi import P, Obj, Context, With, Pipe

text = Pipe(
    "text.txt",
    With( open, Context,
        Obj.read()
    )
)

The previous is equivalent to

with open("text.txt") as f:
    text = f.read()
def With(self, context_manager, *body, **kwargs):
    """
th**
def With(context_manager, *body):
guments**
context_manager**: a [context manager](https://docs.python.org/2/reference/datamodel.html#context-managers) object or valid expression from the DSL that returns a context manager.
*body**: any valid expression of the DSL to be evaluated inside the context. `*body` is interpreted as a tuple so all expression contained are composed.
ith normal python programs you sometimes might want to create a context for a block of code. You normally give a [context manager](https://docs.python.org/2/reference/datamodel.html#context-managers) to the [with](https://docs.python.org/2/reference/compound_stmts.html#the-with-statement) statemente, in Phi you use `P.With` or `phi.With`
ntext**
on's `with` statemente returns a context object through `as` keyword, in the DSL this object can be obtained using the `P.Context` method or the `phi.Context` function.
Examples
from phi import P, Obj, Context, With, Pipe
text = Pipe(
    "text.txt",
    With( open, Context,
        Obj.read()
    )
)
previous is equivalent to
with open("text.txt") as f:
    text = f.read()
    """
    context_f = _parse(context_manager)._f
    body_f = E.Seq(*body)._f
    def g(x, state):
        context, state = context_f(x, state)
        with context as scope:
            with _WithContextManager(scope):
                return body_f(x, state)
    return self.__then__(g, **kwargs)

def Write(

self, *state_args, **state_dict)

See phi.dsl.Expression.Read

def Write(self, *state_args, **state_dict):
    """See `phi.dsl.Expression.Read`"""
    if len(state_dict) + len(state_args) < 1:
        raise Exception("Please include at-least 1 state variable, got {0} and {1}".format(state_args, state_dict))
    if len(state_dict) > 1:
        raise Exception("Please include at-most 1 keyword argument expression, got {0}".format(state_dict))
    if len(state_dict) > 0:
        state_key = next(iter(state_dict.keys()))
        write_expr = state_dict[state_key]
        state_args += (state_key,)
        expr = self >> write_expr
    else:
        expr = self
    def g(x, state):
        update = { key: x for key in state_args }
        state = utils.merge(state, update)
        #side effect for convenience
        _StateContextManager.REFS.update(state)
        return x, state
    return expr.__then__(g)