phi.builder module
The Builder
extends the Expression class and adds methods that enable you to register external functions as methods. You should NOT register functions on the Builder
or PythonBuilder classes, instead you should create a class of your own that extends these and register your functions there. The benefit of having this registration mechanism, instead of just implementing the methods yourself is that you can automate the process to register a collection of existing functions or methods.
See
""" The `phi.builder.Builder` extends the [Expression](https://cgarciae.github.io/phi/dsl.m.html#phi.dsl.Expression) class and adds methods that enable you to register external functions as methods. You should NOT register functions on the `Builder` or [PythonBuilder](https://cgarciae.github.io/phi/python_builder.m.html) classes, instead you should create a class of your own that extends these and register your functions there. The benefit of having this registration mechanism, instead of just implementing the methods yourself is that you can automate the process to register a collection of existing functions or methods. **See** * `phi.builder.Builder.RegisterMethod` * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.PatchAt` """ from __future__ import absolute_import from __future__ import division from __future__ import print_function from __future__ import unicode_literals import inspect from . import utils from .utils import identity import functools from . import dsl ###################### # Helpers ###################### _True = lambda x: True _False = lambda x: False _None = lambda x: None _NoLeadingUnderscore = lambda name: name[0] != "_" ####################### ### Builder ####################### class Builder(dsl.Expression): """ All the public methods of the `Builder`, `Expression` and `Expression` classes start with a capital letter on purpose to avoid name chashes with methods that you might register.""" @classmethod def _RegisterMethod(cls, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True): if wrapped: f = functools.wraps(wrapped)(f) fn_signature = utils.get_method_sig(f) fn_docs = inspect.getdoc(f) name = alias if alias else f.__name__ original_name = f.__name__ if wrapped else original_name if original_name else name f.__name__ = str(name) f.__doc__ = doc if doc else (""" THIS METHOD IS AUTOMATICALLY GENERATED {builder_class}.{name}(*args, **kwargs) It accepts the same arguments as `{library_path}{original_name}`. """ + explanation + """ **{library_path}{original_name}** {fn_docs} """).format(original_name=original_name, name=name, fn_docs=fn_docs, library_path=library_path, builder_class=cls.__name__) if explain else fn_docs if name in cls.__core__: raise Exception("Can't add method '{0}' because its on __core__".format(name)) f = method_type(f) setattr(cls, name, f) @classmethod def RegisterMethod(cls, *args, **kwargs): """ **RegisterMethod** RegisterMethod(f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True) `classmethod` for registering functions as methods of this class. **Arguments** * **f** : the particular function being registered as a method * **library_path** : library from where `f` comes from, unless you pass an empty string, put a period `"."` at the end of the library name. * `alias=None` : alias for the name/method being registered * `original_name=None` : name of the original function, used for documentation purposes. * `doc=None` : complete documentation of the method being registered * `wrapped=None` : if you are registering a function which wraps around another function, pass this other function through `wrapped` to get better documentation, this is specially useful is you register a bunch of functions in a for loop. Please include an `explanation` to tell how the actual function differs from the wrapped one. * `explanation=""` : especify any additional information for the documentation of the method being registered, you can use any of the following format tags within this string and they will be replace latter on: `{original_name}`, `{name}`, `{fn_docs}`, `{library_path}`, `{builder_class}`. * `method_type=identity` : by default its applied but does nothing, you might also want to register functions as `property`, `classmethod`, `staticmethod` * `explain=True` : decide whether or not to show any kind of explanation, its useful to set it to `False` if you are using a `Register*` decorator and will only use the function as a registered method. A main feature of `phi` is that it enables you to integrate your library or even an existing library with the DSL. You can achieve three levels of integration 1. Passing your functions to the DSL. This a very general machanism -since you could actually do everything with python lamdas- but in practice functions often receive multiple parameters. 2. Creating partials with the `Then*` method family. Using this you could integrate any function, but it will add a lot of noise if you use heavily on it. 3. Registering functions as methods of a `Builder` derived class. This produces the most readable code and its the approach you should take if you want to create a Phi-based library or a helper class. While point 3 is the most desirable it has a cost: you need to create your own `phi.builder.Builder`-derived class. This is because SHOULD NOT register functions to existing builders e.g. the `phi.builder.Builder` or [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) provided by phi because that would pollute the `P` object. Instead you should create a custom class that derives from `phi.builder.Builder`, [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) or another custom builder depending on your needs and register your functions to that class. **Examples** Say you have a function on a library called `"my_lib"` def some_fun(obj, arg1, arg2): # code You could use it with the dsl like this from phi import P, Then P.Pipe( input, ... Then(some_fun, arg1, arg2) ... ) assuming the first parameter `obj` is being piped down. However if you do this very often or you are creating a library, you are better off creating a custom class derived from `Builder` or `PythonBuilder` from phi import Builder #or PythonBuilder class MyBuilder(Builder): # or PythonBuilder pass and registering your function as a method. The first way you could do this is by creating a wrapper function for `some_fun` and registering it as a method def some_fun_wrapper(self, arg1, arg2): return self.Then(some_fun, arg1, arg2) MyBuilder.RegisterMethod(some_fun_wrapper, "my_lib.", wrapped=some_fun) Here we basically created a shortcut for the original expression `Then(some_fun, arg1, arg2)`. You could also do this using a decorator @MyBuilder.RegisterMethod("my_lib.", wrapped=some_fun) def some_fun_wrapper(self, arg1, arg2): return self.Then(some_fun, arg1, arg2) However, this is such a common task that we've created the method `Register` to avoid you from having to create the wrapper. With it you could register the function `some_fun` directly as a method like this MyBuilder.Register(some_fun, "my_lib.") or by using a decorator over the original function definition @MyBuilder.Register("my_lib.") def some_fun(obj, arg1, arg2): # code Once done you've done any of the previous approaches you can create a custom global object e.g. `M` and use it instead of/along with `P` M = MyBuilder(lambda x: x) M.Pipe( input, ... M.some_fun(arg1, args) ... ) **Argument position** `phi.builder.Builder.Register` internally uses `phi.builder.Builder.Then`, this is only useful if the object being piped is intended to be passed as the first argument of the function being registered, if this is not the case you could use `phi.builder.Builder.Register2`, `phi.builder.Builder.Register3`, ..., `phi.builder.Builder.Register5` or `phi.builder.Builder.RegisterAt` to set an arbitrary position, these functions will internally use `phi.builder.Builder.Then2`, `phi.builder.Builder.Then3`, ..., `phi.builder.Builder.Then5` or `phi.builder.Builder.ThenAt` respectively. **Wrapping functions** Sometimes you have an existing function that you would like to modify slightly so it plays nicely with the DSL, what you normally do is create a function that wraps around it and passes the arguments to it in a way that is convenient import some_lib @MyBuilder.Register("some_lib.") def some_fun(a, n): return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified When you do this -as a side effect- you loose the original documentation, to avoid this you can use the Registers `wrapped` argument along with the `explanation` argument to clarity the situation import some_lib some_fun_explanation = "However, it differs in that `n` is automatically subtracted `1`" @MyBuilder.Register("some_lib.", wrapped=some_lib.some_fun, explanation=some_fun_explanation) def some_fun(a, n): return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified Now the documentation for `MyBuilder.some_fun` will be a little bit nicer since it includes the original documentation from `some_lib.some_fun`. This behaviour is specially useful if you are wrapping an entire 3rd party library, you usually automate the process iterating over all the funcitions in a for loop. The `phi.builder.Builder.PatchAt` method lets you register and entire module using a few lines of code, however, something you have to do thing more manually and do the iteration yourself. **See Also** * `phi.builder.Builder.PatchAt` * `phi.builder.Builder.RegisterAt` """ unpack_error = True try: f, library_path = args unpack_error = False cls._RegisterMethod(f, library_path, **kwargs) except: if not unpack_error: raise def register_decorator(f): library_path, = args cls._RegisterMethod(f, library_path, **kwargs) return f return register_decorator @classmethod def _RegisterAt(cls, n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None): _wrapped = wrapped if wrapped else f try: @functools.wraps(f) def method(self, *args, **kwargs): kwargs['_return_type'] = _return_type return self.ThenAt(n, f, *args, **kwargs) except: raise all_args, previous_args, last_arg = _make_args_strs(n) explanation = """ However, the 1st argument is omitted, a partial with the rest of the arguments is returned which expects the 1st argument such that {library_path}{original_name}("""+ all_args +"""*args, **kwargs) is equivalent to {builder_class}.{name}("""+ previous_args +"""*args, **kwargs)("""+ last_arg +""") """ + explanation if explain else "" cls.RegisterMethod(method, library_path, alias=alias, original_name=original_name, doc=doc, wrapped=wrapped, explanation=explanation, method_type=method_type, explain=explain) @classmethod def RegisterAt(cls, *args, **kwargs): """ **RegisterAt** RegisterAt(n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None) Most of the time you don't want to register an method as such, that is, you don't care about the `self` builder object, instead you want to register a function that transforms the value being piped down the DSL. For this you can use `RegisterAt` so e.g. def some_fun(obj, arg1, arg2): # code @MyBuilder.RegisterMethod("my_lib.") def some_fun_wrapper(self, arg1, arg2): return self.ThenAt(1, some_fun, arg1, arg2) can be written directly as @MyBuilder.RegisterAt(1, "my_lib.") def some_fun(obj, arg1, arg2): # code For this case you can just use `Register` which is a shortcut for `RegisterAt(1, ...)` @MyBuilder.Register("my_lib.") def some_fun(obj, arg1, arg2): # code **Also See** * `phi.builder.Builder.RegisterMethod` """ unpack_error = True try: n, f, library_path = args unpack_error = False cls._RegisterAt(n, f, library_path, **kwargs) except: if not unpack_error: raise def register_decorator(f): n, library_path = args cls._RegisterAt(n, f, library_path, **kwargs) return f return register_decorator @classmethod def Register0(cls, *args, **kwargs): """ `Register0(...)` is a shortcut for `RegisterAt(0, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(0, *args, **kwargs) @classmethod def Register(cls, *args, **kwargs): """ `Register(...)` is a shortcut for `RegisterAt(1, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(1, *args, **kwargs) @classmethod def Register2(cls, *args, **kwargs): """ `Register2(...)` is a shortcut for `RegisterAt(2, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(2, *args, **kwargs) @classmethod def Register3(cls, *args, **kwargs): """ `Register3(...)` is a shortcut for `RegisterAt(3, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(3, *args, **kwargs) @classmethod def Register4(cls, *args, **kwargs): """ `Register4(...)` is a shortcut for `RegisterAt(4, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(4, *args, **kwargs) @classmethod def Register5(cls, *args, **kwargs): """ `Register5(...)` is a shortcut for `RegisterAt(5, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(5, *args, **kwargs) @classmethod def PatchAt(cls, n, module, method_wrapper=None, module_alias=None, method_name_modifier=utils.identity, blacklist_predicate=_False, whitelist_predicate=_True, return_type_predicate=_None, getmembers_predicate=inspect.isfunction, admit_private=False, explanation=""): """ This classmethod lets you easily patch all of functions/callables from a module or class as methods a Builder class. **Arguments** * **n** : the position the the object being piped will take in the arguments when the function being patched is applied. See `RegisterMethod` and `ThenAt`. * **module** : a module or class from which the functions/methods/callables will be taken. * `module_alias = None` : an optional alias for the module used for documentation purposes. * `method_name_modifier = lambda f_name: None` : a function that can modify the name of the method will take. If `None` the name of the function will be used. * `blacklist_predicate = lambda f_name: name[0] != "_"` : A predicate that determines which functions are banned given their name. By default it excludes all function whose name start with `'_'`. `blacklist_predicate` can also be of type list, in which case all names contained in this list will be banned. * `whitelist_predicate = lambda f_name: True` : A predicate that determines which functions are admitted given their name. By default it include any function. `whitelist_predicate` can also be of type list, in which case only names contained in this list will be admitted. You can use both `blacklist_predicate` and `whitelist_predicate` at the same time. * `return_type_predicate = lambda f_name: None` : a predicate that determines the `_return_type` of the Builder. By default it will always return `None`. See `phi.builder.Builder.ThenAt`. * `getmembers_predicate = inspect.isfunction` : a predicate that determines what type of elements/members will be fetched by the `inspect` module, defaults to [inspect.isfunction](https://docs.python.org/2/library/inspect.html#inspect.isfunction). See [getmembers](https://docs.python.org/2/library/inspect.html#inspect.getmembers). **Examples** Lets patch ALL the main functions from numpy into a custom builder! from phi import PythonBuilder #or Builder import numpy as np class NumpyBuilder(PythonBuilder): #or Builder "A Builder for numpy functions!" pass NumpyBuilder.PatchAt(1, np) N = NumpyBuilder(lambda x: x) Thats it! Although a serious patch would involve filtering out functions that don't take arrays. Another common task would be to use `NumpyBuilder.PatchAt(2, ...)` (`PatchAt(n, ..)` in general) when convenient to send the object being pipe to the relevant argument of the function. The previous is usually done with and a combination of `whitelist_predicate`s and `blacklist_predicate`s on `PatchAt(1, ...)` and `PatchAt(2, ...)` to filter or include the approriate functions on each kind of patch. Given the previous code we could now do import numpy as np x = np.array([[1,2],[3,4]]) y = np.array([[5,6],[7,8]]) z = N.Pipe( x, N .dot(y) .add(x) .transpose() .sum(axis=1) ) Which is strictly equivalent to import numpy as np x = np.array([[1,2],[3,4]]) y = np.array([[5,6],[7,8]]) z = np.dot(x, y) z = np.add(z, x) z = np.transpose(z) z = np.sum(z, axis=1) The thing to notice is that with the `NumpyBuilder` we avoid the repetitive and needless passing and reassigment of the `z` variable, this removes a lot of noise from our code. """ _rtp = return_type_predicate return_type_predicate = (lambda x: _rtp) if inspect.isclass(_rtp) and issubclass(_rtp, Builder) else _rtp module_name = module_alias if module_alias else module.__name__ + '.' patch_members = _get_patch_members(module, blacklist_predicate=blacklist_predicate, whitelist_predicate=whitelist_predicate, getmembers_predicate=getmembers_predicate, admit_private=admit_private) if type(whitelist_predicate) is list and "convolution2d" in whitelist_predicate: import ipdb #ipdb.set_trace() for name, f in patch_members: wrapped = None if method_wrapper: g = method_wrapper(f) wrapped = f else: g = f cls.RegisterAt(n, g, module_name, wrapped=wrapped, _return_type=return_type_predicate(name), alias=method_name_modifier(name), explanation=explanation) Builder.__core__ = [ name for name, f in inspect.getmembers(Builder, inspect.ismethod) ] ####################### # Helper functions ####################### def _make_args_strs(n): if n == 0: return "", "", "x" n += 1 all_args = [ 'x' + str(i) for i in range(1, n) ] last = all_args[n-2] previous = all_args[:n-2] return ", ".join(all_args + [""]), ", ".join(previous + [""]), last def _get_patch_members(module, blacklist_predicate=_NoLeadingUnderscore, whitelist_predicate=_True, _return_type=None, getmembers_predicate=inspect.isfunction, admit_private=False): if type(whitelist_predicate) is list: whitelist = whitelist_predicate whitelist_predicate = lambda x: x in whitelist if type(blacklist_predicate) is list: blacklist = blacklist_predicate blacklist_predicate = lambda x: x in blacklist or '_' == x[0] if not admit_private else False return [ (name, f) for (name, f) in inspect.getmembers(module, getmembers_predicate) if whitelist_predicate(name) and not blacklist_predicate(name) ]
Module variables
var name
Classes
class Builder
All the public methods of the Builder
, Expression
and Expression
classes start with a capital letter on purpose to avoid name chashes with methods that you might register.
class Builder(dsl.Expression): """ All the public methods of the `Builder`, `Expression` and `Expression` classes start with a capital letter on purpose to avoid name chashes with methods that you might register.""" @classmethod def _RegisterMethod(cls, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True): if wrapped: f = functools.wraps(wrapped)(f) fn_signature = utils.get_method_sig(f) fn_docs = inspect.getdoc(f) name = alias if alias else f.__name__ original_name = f.__name__ if wrapped else original_name if original_name else name f.__name__ = str(name) f.__doc__ = doc if doc else (""" THIS METHOD IS AUTOMATICALLY GENERATED {builder_class}.{name}(*args, **kwargs) It accepts the same arguments as `{library_path}{original_name}`. """ + explanation + """ **{library_path}{original_name}** {fn_docs} """).format(original_name=original_name, name=name, fn_docs=fn_docs, library_path=library_path, builder_class=cls.__name__) if explain else fn_docs if name in cls.__core__: raise Exception("Can't add method '{0}' because its on __core__".format(name)) f = method_type(f) setattr(cls, name, f) @classmethod def RegisterMethod(cls, *args, **kwargs): """ **RegisterMethod** RegisterMethod(f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True) `classmethod` for registering functions as methods of this class. **Arguments** * **f** : the particular function being registered as a method * **library_path** : library from where `f` comes from, unless you pass an empty string, put a period `"."` at the end of the library name. * `alias=None` : alias for the name/method being registered * `original_name=None` : name of the original function, used for documentation purposes. * `doc=None` : complete documentation of the method being registered * `wrapped=None` : if you are registering a function which wraps around another function, pass this other function through `wrapped` to get better documentation, this is specially useful is you register a bunch of functions in a for loop. Please include an `explanation` to tell how the actual function differs from the wrapped one. * `explanation=""` : especify any additional information for the documentation of the method being registered, you can use any of the following format tags within this string and they will be replace latter on: `{original_name}`, `{name}`, `{fn_docs}`, `{library_path}`, `{builder_class}`. * `method_type=identity` : by default its applied but does nothing, you might also want to register functions as `property`, `classmethod`, `staticmethod` * `explain=True` : decide whether or not to show any kind of explanation, its useful to set it to `False` if you are using a `Register*` decorator and will only use the function as a registered method. A main feature of `phi` is that it enables you to integrate your library or even an existing library with the DSL. You can achieve three levels of integration 1. Passing your functions to the DSL. This a very general machanism -since you could actually do everything with python lamdas- but in practice functions often receive multiple parameters. 2. Creating partials with the `Then*` method family. Using this you could integrate any function, but it will add a lot of noise if you use heavily on it. 3. Registering functions as methods of a `Builder` derived class. This produces the most readable code and its the approach you should take if you want to create a Phi-based library or a helper class. While point 3 is the most desirable it has a cost: you need to create your own `phi.builder.Builder`-derived class. This is because SHOULD NOT register functions to existing builders e.g. the `phi.builder.Builder` or [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) provided by phi because that would pollute the `P` object. Instead you should create a custom class that derives from `phi.builder.Builder`, [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) or another custom builder depending on your needs and register your functions to that class. **Examples** Say you have a function on a library called `"my_lib"` def some_fun(obj, arg1, arg2): # code You could use it with the dsl like this from phi import P, Then P.Pipe( input, ... Then(some_fun, arg1, arg2) ... ) assuming the first parameter `obj` is being piped down. However if you do this very often or you are creating a library, you are better off creating a custom class derived from `Builder` or `PythonBuilder` from phi import Builder #or PythonBuilder class MyBuilder(Builder): # or PythonBuilder pass and registering your function as a method. The first way you could do this is by creating a wrapper function for `some_fun` and registering it as a method def some_fun_wrapper(self, arg1, arg2): return self.Then(some_fun, arg1, arg2) MyBuilder.RegisterMethod(some_fun_wrapper, "my_lib.", wrapped=some_fun) Here we basically created a shortcut for the original expression `Then(some_fun, arg1, arg2)`. You could also do this using a decorator @MyBuilder.RegisterMethod("my_lib.", wrapped=some_fun) def some_fun_wrapper(self, arg1, arg2): return self.Then(some_fun, arg1, arg2) However, this is such a common task that we've created the method `Register` to avoid you from having to create the wrapper. With it you could register the function `some_fun` directly as a method like this MyBuilder.Register(some_fun, "my_lib.") or by using a decorator over the original function definition @MyBuilder.Register("my_lib.") def some_fun(obj, arg1, arg2): # code Once done you've done any of the previous approaches you can create a custom global object e.g. `M` and use it instead of/along with `P` M = MyBuilder(lambda x: x) M.Pipe( input, ... M.some_fun(arg1, args) ... ) **Argument position** `phi.builder.Builder.Register` internally uses `phi.builder.Builder.Then`, this is only useful if the object being piped is intended to be passed as the first argument of the function being registered, if this is not the case you could use `phi.builder.Builder.Register2`, `phi.builder.Builder.Register3`, ..., `phi.builder.Builder.Register5` or `phi.builder.Builder.RegisterAt` to set an arbitrary position, these functions will internally use `phi.builder.Builder.Then2`, `phi.builder.Builder.Then3`, ..., `phi.builder.Builder.Then5` or `phi.builder.Builder.ThenAt` respectively. **Wrapping functions** Sometimes you have an existing function that you would like to modify slightly so it plays nicely with the DSL, what you normally do is create a function that wraps around it and passes the arguments to it in a way that is convenient import some_lib @MyBuilder.Register("some_lib.") def some_fun(a, n): return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified When you do this -as a side effect- you loose the original documentation, to avoid this you can use the Registers `wrapped` argument along with the `explanation` argument to clarity the situation import some_lib some_fun_explanation = "However, it differs in that `n` is automatically subtracted `1`" @MyBuilder.Register("some_lib.", wrapped=some_lib.some_fun, explanation=some_fun_explanation) def some_fun(a, n): return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified Now the documentation for `MyBuilder.some_fun` will be a little bit nicer since it includes the original documentation from `some_lib.some_fun`. This behaviour is specially useful if you are wrapping an entire 3rd party library, you usually automate the process iterating over all the funcitions in a for loop. The `phi.builder.Builder.PatchAt` method lets you register and entire module using a few lines of code, however, something you have to do thing more manually and do the iteration yourself. **See Also** * `phi.builder.Builder.PatchAt` * `phi.builder.Builder.RegisterAt` """ unpack_error = True try: f, library_path = args unpack_error = False cls._RegisterMethod(f, library_path, **kwargs) except: if not unpack_error: raise def register_decorator(f): library_path, = args cls._RegisterMethod(f, library_path, **kwargs) return f return register_decorator @classmethod def _RegisterAt(cls, n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None): _wrapped = wrapped if wrapped else f try: @functools.wraps(f) def method(self, *args, **kwargs): kwargs['_return_type'] = _return_type return self.ThenAt(n, f, *args, **kwargs) except: raise all_args, previous_args, last_arg = _make_args_strs(n) explanation = """ However, the 1st argument is omitted, a partial with the rest of the arguments is returned which expects the 1st argument such that {library_path}{original_name}("""+ all_args +"""*args, **kwargs) is equivalent to {builder_class}.{name}("""+ previous_args +"""*args, **kwargs)("""+ last_arg +""") """ + explanation if explain else "" cls.RegisterMethod(method, library_path, alias=alias, original_name=original_name, doc=doc, wrapped=wrapped, explanation=explanation, method_type=method_type, explain=explain) @classmethod def RegisterAt(cls, *args, **kwargs): """ **RegisterAt** RegisterAt(n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None) Most of the time you don't want to register an method as such, that is, you don't care about the `self` builder object, instead you want to register a function that transforms the value being piped down the DSL. For this you can use `RegisterAt` so e.g. def some_fun(obj, arg1, arg2): # code @MyBuilder.RegisterMethod("my_lib.") def some_fun_wrapper(self, arg1, arg2): return self.ThenAt(1, some_fun, arg1, arg2) can be written directly as @MyBuilder.RegisterAt(1, "my_lib.") def some_fun(obj, arg1, arg2): # code For this case you can just use `Register` which is a shortcut for `RegisterAt(1, ...)` @MyBuilder.Register("my_lib.") def some_fun(obj, arg1, arg2): # code **Also See** * `phi.builder.Builder.RegisterMethod` """ unpack_error = True try: n, f, library_path = args unpack_error = False cls._RegisterAt(n, f, library_path, **kwargs) except: if not unpack_error: raise def register_decorator(f): n, library_path = args cls._RegisterAt(n, f, library_path, **kwargs) return f return register_decorator @classmethod def Register0(cls, *args, **kwargs): """ `Register0(...)` is a shortcut for `RegisterAt(0, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(0, *args, **kwargs) @classmethod def Register(cls, *args, **kwargs): """ `Register(...)` is a shortcut for `RegisterAt(1, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(1, *args, **kwargs) @classmethod def Register2(cls, *args, **kwargs): """ `Register2(...)` is a shortcut for `RegisterAt(2, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(2, *args, **kwargs) @classmethod def Register3(cls, *args, **kwargs): """ `Register3(...)` is a shortcut for `RegisterAt(3, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(3, *args, **kwargs) @classmethod def Register4(cls, *args, **kwargs): """ `Register4(...)` is a shortcut for `RegisterAt(4, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(4, *args, **kwargs) @classmethod def Register5(cls, *args, **kwargs): """ `Register5(...)` is a shortcut for `RegisterAt(5, ...)` **Also See** * `phi.builder.Builder.RegisterAt` * `phi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(5, *args, **kwargs) @classmethod def PatchAt(cls, n, module, method_wrapper=None, module_alias=None, method_name_modifier=utils.identity, blacklist_predicate=_False, whitelist_predicate=_True, return_type_predicate=_None, getmembers_predicate=inspect.isfunction, admit_private=False, explanation=""): """ This classmethod lets you easily patch all of functions/callables from a module or class as methods a Builder class. **Arguments** * **n** : the position the the object being piped will take in the arguments when the function being patched is applied. See `RegisterMethod` and `ThenAt`. * **module** : a module or class from which the functions/methods/callables will be taken. * `module_alias = None` : an optional alias for the module used for documentation purposes. * `method_name_modifier = lambda f_name: None` : a function that can modify the name of the method will take. If `None` the name of the function will be used. * `blacklist_predicate = lambda f_name: name[0] != "_"` : A predicate that determines which functions are banned given their name. By default it excludes all function whose name start with `'_'`. `blacklist_predicate` can also be of type list, in which case all names contained in this list will be banned. * `whitelist_predicate = lambda f_name: True` : A predicate that determines which functions are admitted given their name. By default it include any function. `whitelist_predicate` can also be of type list, in which case only names contained in this list will be admitted. You can use both `blacklist_predicate` and `whitelist_predicate` at the same time. * `return_type_predicate = lambda f_name: None` : a predicate that determines the `_return_type` of the Builder. By default it will always return `None`. See `phi.builder.Builder.ThenAt`. * `getmembers_predicate = inspect.isfunction` : a predicate that determines what type of elements/members will be fetched by the `inspect` module, defaults to [inspect.isfunction](https://docs.python.org/2/library/inspect.html#inspect.isfunction). See [getmembers](https://docs.python.org/2/library/inspect.html#inspect.getmembers). **Examples** Lets patch ALL the main functions from numpy into a custom builder! from phi import PythonBuilder #or Builder import numpy as np class NumpyBuilder(PythonBuilder): #or Builder "A Builder for numpy functions!" pass NumpyBuilder.PatchAt(1, np) N = NumpyBuilder(lambda x: x) Thats it! Although a serious patch would involve filtering out functions that don't take arrays. Another common task would be to use `NumpyBuilder.PatchAt(2, ...)` (`PatchAt(n, ..)` in general) when convenient to send the object being pipe to the relevant argument of the function. The previous is usually done with and a combination of `whitelist_predicate`s and `blacklist_predicate`s on `PatchAt(1, ...)` and `PatchAt(2, ...)` to filter or include the approriate functions on each kind of patch. Given the previous code we could now do import numpy as np x = np.array([[1,2],[3,4]]) y = np.array([[5,6],[7,8]]) z = N.Pipe( x, N .dot(y) .add(x) .transpose() .sum(axis=1) ) Which is strictly equivalent to import numpy as np x = np.array([[1,2],[3,4]]) y = np.array([[5,6],[7,8]]) z = np.dot(x, y) z = np.add(z, x) z = np.transpose(z) z = np.sum(z, axis=1) The thing to notice is that with the `NumpyBuilder` we avoid the repetitive and needless passing and reassigment of the `z` variable, this removes a lot of noise from our code. """ _rtp = return_type_predicate return_type_predicate = (lambda x: _rtp) if inspect.isclass(_rtp) and issubclass(_rtp, Builder) else _rtp module_name = module_alias if module_alias else module.__name__ + '.' patch_members = _get_patch_members(module, blacklist_predicate=blacklist_predicate, whitelist_predicate=whitelist_predicate, getmembers_predicate=getmembers_predicate, admit_private=admit_private) if type(whitelist_predicate) is list and "convolution2d" in whitelist_predicate: import ipdb #ipdb.set_trace() for name, f in patch_members: wrapped = None if method_wrapper: g = method_wrapper(f) wrapped = f else: g = f cls.RegisterAt(n, g, module_name, wrapped=wrapped, _return_type=return_type_predicate(name), alias=method_name_modifier(name), explanation=explanation)
Ancestors (in MRO)
- Builder
- phi.dsl.Expression
- __builtin__.object
Static methods
def Context(
*args)
Builder Core. Also available as a global function as phi.Context
.
Returns the context object of the current dsl.With
statemente.
Arguments
- *args: By design
Context
accepts any number of arguments and completely ignores them.
This is a classmethod and it doesnt return a Builder
/Expression
by design so it can be called directly:
from phi import P, Context, Obj def read_file(z): f = Context() return f.read() lines = P.Pipe( "text.txt", P.With( open, read_file, Obj.split("\n") ) )
Here we called Context
with no arguments to get the context back, however, since you can also give this function an argument (which it will ignore) it can be passed to the DSL so we can rewrite the previous as:
from phi import P, Context, Obj lines = P.Pipe( "text.txt", P.With( open, Context, # f Obj.read() Obj.split("\n") ) )
Context
yields an exception when used outside of a With
block.
Also see
@staticmethod def Context(*args): """ ilder Core**. Also available as a global function as `phi.Context`. rns the context object of the current `dsl.With` statemente. guments** *args**: By design `Context` accepts any number of arguments and completely ignores them. is a classmethod and it doesnt return a `Builder`/`Expression` by design so it can be called directly: from phi import P, Context, Obj def read_file(z): f = Context() return f.read() lines = P.Pipe( "text.txt", P.With( open, read_file, Obj.split("\\n") ) ) we called `Context` with no arguments to get the context back, however, since you can also give this function an argument (which it will ignore) it can be passed to the DSL so we can rewrite the previous as: from phi import P, Context, Obj lines = P.Pipe( "text.txt", P.With( open, Context, # f Obj.read() Obj.split("\\n") ) ) text` yields an exception when used outside of a `With` block. so see** hi.builder.Builder.Obj` sl](https://cgarciae.github.io/phi/dsl.m.html) """ if _WithContextManager.WITH_GLOBAL_CONTEXT is utils.NO_VALUE: raise Exception("Cannot use 'Context' outside of a 'With' block") return _WithContextManager.WITH_GLOBAL_CONTEXT
Instance variables
var Obj
Obj
is a property
that returns an object that defines the __getattr__
method which when called helps you create a partial that emulates a method call. The following expression
Obj.some_method(x1, x2, ...)
is equivalent to
lambda obj: obj.some_method(x1, x2, ...)
Examples
from phi import P, Obj assert "hello world" == P.Pipe( " HELLO HELLO {0} ", Obj.format("WORLD"), # " HELLO HELLO WORLD " Obj.strip(), # "HELLO HELLO WORLD" Obj.lower() # "hello hello world" Obj.split(' ') # ["hello", "hello", "world"] Obj.count("hello") # 2 )
Also see
var Read
Giving names and saving parts of your computation to use then latter is useful to say the least. In Phi the expression
Write(x = expr)
creates a reference x
given the value of expr
which you can call latter. To read the previous you would use any of the following expressions
Read('x') Read.x
Example
Lets see a common situation where you would use this
from phi import P, List, Seq, Read, Write result = P.Pipe( input, Write(ref = f1), f2, List( f3 , Seq( Read('ref'), f4 ) ) )
Here you save the value outputed by fun_1
and the load it as the initial value of the second branch. In normal python the previous would be almost equivalent to
x = f1(input) ref = x x = f2(x) result = [ f3(x) , f4(ref) ]
var Rec
Rec
is a property
that returns an object that defines the __getattr__
and __getitem__
methods which when called help you create lambdas that emulates a field access. The following expression
Rec.some_field
is equivalent to
lambda rec: rec.some_field
Examples
from phi import P, Obj, Rec class Point(object): def __init__(self, x, y): self.x = x self.y = y def flip_cords(self): y = self.y self.y = self.x self.x = y assert 4 == P.Pipe( Point(1, 2), # point(x=1, y=2) Obj.flip_cords(), # point(x=2, y=1) Rec.x, # point.x = 2 P * 2 # 2 * 2 = 4 )
Also see
var Ref
Returns an object that helps you to inmediatly create and read references.
Creating Refences
You can manually create a Ref outside the DSL using Ref
and then pass to as/to a Read or Write expression. Here is a contrived example
from phi import P r = P.Ref('r') assert [600, 3, 6] == P.Pipe( 2, P + 1, {'a'}, # a = 2 + 1 = 3 P * 2, {'b'}, # b = 3 * 2 = 6 P * 100, {'c', r }, # c = r = 6 * 100 = 600 ['c', 'a', 'b'] ) assert r() == 600
Reading Refences from the Current Context
While the expression Read.a
with return a function that will discard its argument and return the value of the reference x
in the current context, the expression Ref.x
will return the value inmediatly, this is useful when using it inside pyton lambdas.
Read.x(None) <=> Ref.x
As an example
from phi import P, Obj, Ref assert {'a': 97, 'b': 98, 'c': 99} == P.Pipe( "a b c", Obj .split(' ').Write.keys # keys = ['a', 'b', 'c'] .map(ord), # [ord('a'), ord('b'), ord('c')] == [97, 98, 99] lambda it: zip(Ref.keys, it), # [('a', 97), ('b', 98), ('c', 99)] dict # {'a': 97, 'b': 98, 'c': 99} )
Methods
def __init__(
self, f=<function state_identity at 0x7fea7e92eaa0>)
def __init__(self, f=utils.state_identity): self._f = f
def Dict(
self, **branches)
def Dict(self, **branches): gs = { key : _parse(value)._f for key, value in branches.items() } def h(x, state): ys = {} for key, g in gs.items(): y, state = g(x, state) ys[key] = y return _RecordObject(**ys), state return self.__then__(h)
def Elif(
self, condition, *then, **kwargs)
See phi.dsl.Expression.If
def Elif(self, condition, *then, **kwargs): """See `phi.dsl.Expression.If`""" root = self._root ast = self._ast cond_f = _parse(condition)._f then_f = E.Seq(*then)._f else_f = utils.state_identity next_else = (cond_f, then_f, else_f) ast = _add_else(ast, next_else) g = _compile_if(ast) expr = root.__then__(g, **kwargs) expr._ast = ast expr._root = root return expr
def Else(
self, *Else, **kwargs)
See phi.dsl.Expression.If
def Else(self, *Else, **kwargs): """See `phi.dsl.Expression.If`""" root = self._root ast = self._ast next_else = E.Seq(*Else)._f ast = _add_else(ast, next_else) g = _compile_if(ast) return root.__then__(g, **kwargs)
def F(
self, expr)
def F(self, expr): return self >> expr
def If(
self, condition, *then, **kwargs)
If
If(Predicate, *Then)
Having conditionals expressions a necesity in every language, Phi includes the If
expression for such a purpose.
Arguments
- Predicate : a predicate expression uses to determine if the
Then
orElse
branches should be used. - *Then : an expression to be excecuted if the
Predicate
yieldsTrue
, since this parameter is variadic you can stack expression and they will be interpreted as a tuplephi.dsl.Seq
.
This class also includes the Elif
and Else
methods which let you write branched conditionals in sequence, however the following rules apply
- If no branch is entered the whole expression behaves like the identity
Elif
can only be used after anIf
or anotherElif
expression- Many
Elif
expressions can be stacked sequentially Else
can only be used after anIf
orElif
expression
Examples
from phi import P, If assert "Between 2 and 10" == P.Pipe( 5, If(P > 10, "Greater than 10" ).Elif(P < 2, "Less than 2" ).Else( "Between 2 and 10" ) )
def If(self, condition, *then, **kwargs): """ ** If(Predicate, *Then) ng conditionals expressions a necesity in every language, Phi includes the `If` expression for such a purpose. guments** Predicate** : a predicate expression uses to determine if the `Then` or `Else` branches should be used. *Then** : an expression to be excecuted if the `Predicate` yields `True`, since this parameter is variadic you can stack expression and they will be interpreted as a tuple `phi.dsl.Seq`. class also includes the `Elif` and `Else` methods which let you write branched conditionals in sequence, however the following rules apply no branch is entered the whole expression behaves like the identity lif` can only be used after an `If` or another `Elif` expression ny `Elif` expressions can be stacked sequentially lse` can only be used after an `If` or `Elif` expression xamples ** from phi import P, If assert "Between 2 and 10" == P.Pipe( 5, If(P > 10, "Greater than 10" ).Elif(P < 2, "Less than 2" ).Else( "Between 2 and 10" ) ) """ cond_f = _parse(condition)._f then_f = E.Seq(*then)._f else_f = utils.state_identity ast = (cond_f, then_f, else_f) g = _compile_if(ast) expr = self.__then__(g, **kwargs) expr._ast = ast expr._root = self return expr
def List(
self, *branches, **kwargs)
While Seq
is sequential, phi.dsl.Expression.List
allows you to split the computation and get back a list with the result of each path. While the list literal should be the most incarnation of this expresion, it can actually be any iterable (implements __iter__
) that is not a tuple and yields a valid expresion.
The expression
k = List(f, g)
is equivalent to
k = lambda x: [ f(x), g(x) ]
In general, the following rules apply after compilation:
General Branching
List(f0, f1, ..., fn)
is equivalent to
lambda x: [ f0(x), f1(x), ..., fn(x) ]
Composing & Branching
It is interesting to see how braching interacts with composing. The expression
Seq(f, List(g, h))
is almost equivalent to
List( Seq(f, g), Seq(f, h) )
As you see its as if f
where distributed over the List. We say almost because their implementation is different
def _lambda(x): x = f(x) return [ g(x), h(x) ]
vs
lambda x: [ g(f(x)), h(f(x)) ]
As you see f
is only executed once in the first one. Both should yield the same result if f
is a pure function.
Examples
form phi import P, List avg_word_length = P.Pipe( "1 22 333", lambda s: s.split(' '), # ['1', '22', '333'] lambda l: map(len, l), # [1, 2, 3] List( sum # 1 + 2 + 3 == 6 , len # len([1, 2, 3]) == 3 ), lambda l: l[0] / l[1] # sum / len == 6 / 3 == 2 ) assert avg_word_length == 2
The previous could also be done more briefly like this
form phi import P, Obj, List avg_word_length = P.Pipe( "1 22 333", Obj .split(' ') # ['1', '22', '333'] .map(len) # [1, 2, 3] .List( sum #sum([1, 2, 3]) == 6 , len #len([1, 2, 3]) == 3 ), P[0] / P[1] #6 / 3 == 2 ) assert avg_word_length == 2
In the example above the last expression
P[0] / P[1]
works for a couple of reasons
- The previous expression returns a list
- In general the expression
P[x]
compiles to a function with the formlambda obj: obj[x]
-
The class
Expression
(the class from which the objectP
inherits) overrides most operators to create functions easily. For example, the expression(P * 2) / (P + 1)
compile to a function of the form
lambda x: (x * 2) / (x + 1)
Check out the documentatio for Phi lambdas.
def List(self, *branches, **kwargs): """ e `Seq` is sequential, `phi.dsl.Expression.List` allows you to split the computation and get back a list with the result of each path. While the list literal should be the most incarnation of this expresion, it can actually be any iterable (implements `__iter__`) that is not a tuple and yields a valid expresion. expression k = List(f, g) quivalent to k = lambda x: [ f(x), g(x) ] eneral, the following rules apply after compilation: neral Branching** List(f0, f1, ..., fn) quivalent to lambda x: [ f0(x), f1(x), ..., fn(x) ] mposing & Branching** s interesting to see how braching interacts with composing. The expression Seq(f, List(g, h)) almost* equivalent to List( Seq(f, g), Seq(f, h) ) ou see its as if `f` where distributed over the List. We say *almost* because their implementation is different def _lambda(x): x = f(x) return [ g(x), h(x) ] lambda x: [ g(f(x)), h(f(x)) ] ou see `f` is only executed once in the first one. Both should yield the same result if `f` is a pure function. Examples form phi import P, List avg_word_length = P.Pipe( "1 22 333", lambda s: s.split(' '), # ['1', '22', '333'] lambda l: map(len, l), # [1, 2, 3] List( sum # 1 + 2 + 3 == 6 , len # len([1, 2, 3]) == 3 ), lambda l: l[0] / l[1] # sum / len == 6 / 3 == 2 ) assert avg_word_length == 2 previous could also be done more briefly like this form phi import P, Obj, List avg_word_length = P.Pipe( "1 22 333", Obj .split(' ') # ['1', '22', '333'] .map(len) # [1, 2, 3] .List( sum #sum([1, 2, 3]) == 6 , len #len([1, 2, 3]) == 3 ), P[0] / P[1] #6 / 3 == 2 ) assert avg_word_length == 2 he example above the last expression P[0] / P[1] s for a couple of reasons he previous expression returns a list n general the expression `P[x]` compiles to a function with the form `lambda obj: obj[x]` he class `Expression` (the class from which the object `P` inherits) overrides most operators to create functions easily. For example, the expression (P * 2) / (P + 1) ile to a function of the form lambda x: (x * 2) / (x + 1) k out the documentatio for Phi [lambdas](https://cgarciae.github.io/phi/lambdas.m.html). """ gs = [ _parse(code)._f for code in branches ] def h(x, state): ys = [] for g in gs: y, state = g(x, state) ys.append(y) return (ys, state) return self.__then__(h, **kwargs)
def PatchAt(
cls, n, module, method_wrapper=None, module_alias=None, method_name_modifier=<function identity at 0x7fea7e92ea28>, blacklist_predicate=<function <lambda> at 0x7fea7e8e27d0>, whitelist_predicate=<function <lambda> at 0x7fea7e8e2758>, return_type_predicate=<function <lambda> at 0x7fea7e8e2848>, getmembers_predicate=<function isfunction at 0x7fea7fdff488>, admit_private=False, explanation=u'')
This classmethod lets you easily patch all of functions/callables from a module or class as methods a Builder class.
Arguments
- n : the position the the object being piped will take in the arguments when the function being patched is applied. See
RegisterMethod
andThenAt
. - module : a module or class from which the functions/methods/callables will be taken.
module_alias = None
: an optional alias for the module used for documentation purposes.method_name_modifier = lambda f_name: None
: a function that can modify the name of the method will take. IfNone
the name of the function will be used.blacklist_predicate = lambda f_name: name[0] != "_"
: A predicate that determines which functions are banned given their name. By default it excludes all function whose name start with'_'
.blacklist_predicate
can also be of type list, in which case all names contained in this list will be banned.whitelist_predicate = lambda f_name: True
: A predicate that determines which functions are admitted given their name. By default it include any function.whitelist_predicate
can also be of type list, in which case only names contained in this list will be admitted. You can use bothblacklist_predicate
andwhitelist_predicate
at the same time.return_type_predicate = lambda f_name: None
: a predicate that determines the_return_type
of the Builder. By default it will always returnNone
. SeeThenAt
.getmembers_predicate = inspect.isfunction
: a predicate that determines what type of elements/members will be fetched by theinspect
module, defaults to inspect.isfunction. See getmembers.
Examples
Lets patch ALL the main functions from numpy into a custom builder!
from phi import PythonBuilder #or Builder import numpy as np class NumpyBuilder(PythonBuilder): #or Builder "A Builder for numpy functions!" pass NumpyBuilder.PatchAt(1, np) N = NumpyBuilder(lambda x: x)
Thats it! Although a serious patch would involve filtering out functions that don't take arrays. Another common task would be to use NumpyBuilder.PatchAt(2, ...)
(PatchAt(n, ..)
in general) when convenient to send the object being pipe to the relevant argument of the function. The previous is usually done with and a combination of whitelist_predicate
s and blacklist_predicate
s on PatchAt(1, ...)
and PatchAt(2, ...)
to filter or include the approriate functions on each kind of patch. Given the previous code we could now do
import numpy as np x = np.array([[1,2],[3,4]]) y = np.array([[5,6],[7,8]]) z = N.Pipe( x, N .dot(y) .add(x) .transpose() .sum(axis=1) )
Which is strictly equivalent to
import numpy as np x = np.array([[1,2],[3,4]]) y = np.array([[5,6],[7,8]]) z = np.dot(x, y) z = np.add(z, x) z = np.transpose(z) z = np.sum(z, axis=1)
The thing to notice is that with the NumpyBuilder
we avoid the repetitive and needless passing and reassigment of the z
variable, this removes a lot of noise from our code.
@classmethod def PatchAt(cls, n, module, method_wrapper=None, module_alias=None, method_name_modifier=utils.identity, blacklist_predicate=_False, whitelist_predicate=_True, return_type_predicate=_None, getmembers_predicate=inspect.isfunction, admit_private=False, explanation=""): """ classmethod lets you easily patch all of functions/callables from a module or class as methods a Builder class. guments** n** : the position the the object being piped will take in the arguments when the function being patched is applied. See `RegisterMethod` and `ThenAt`. module** : a module or class from which the functions/methods/callables will be taken. odule_alias = None` : an optional alias for the module used for documentation purposes. ethod_name_modifier = lambda f_name: None` : a function that can modify the name of the method will take. If `None` the name of the function will be used. lacklist_predicate = lambda f_name: name[0] != "_"` : A predicate that determines which functions are banned given their name. By default it excludes all function whose name start with `'_'`. `blacklist_predicate` can also be of type list, in which case all names contained in this list will be banned. hitelist_predicate = lambda f_name: True` : A predicate that determines which functions are admitted given their name. By default it include any function. `whitelist_predicate` can also be of type list, in which case only names contained in this list will be admitted. You can use both `blacklist_predicate` and `whitelist_predicate` at the same time. eturn_type_predicate = lambda f_name: None` : a predicate that determines the `_return_type` of the Builder. By default it will always return `None`. See `phi.builder.Builder.ThenAt`. etmembers_predicate = inspect.isfunction` : a predicate that determines what type of elements/members will be fetched by the `inspect` module, defaults to [inspect.isfunction](https://docs.python.org/2/library/inspect.html#inspect.isfunction). See [getmembers](https://docs.python.org/2/library/inspect.html#inspect.getmembers). amples** patch ALL the main functions from numpy into a custom builder! from phi import PythonBuilder #or Builder import numpy as np class NumpyBuilder(PythonBuilder): #or Builder "A Builder for numpy functions!" pass NumpyBuilder.PatchAt(1, np) N = NumpyBuilder(lambda x: x) s it! Although a serious patch would involve filtering out functions that don't take arrays. Another common task would be to use `NumpyBuilder.PatchAt(2, ...)` (`PatchAt(n, ..)` in general) when convenient to send the object being pipe to the relevant argument of the function. The previous is usually done with and a combination of `whitelist_predicate`s and `blacklist_predicate`s on `PatchAt(1, ...)` and `PatchAt(2, ...)` to filter or include the approriate functions on each kind of patch. Given the previous code we could now do import numpy as np x = np.array([[1,2],[3,4]]) y = np.array([[5,6],[7,8]]) z = N.Pipe( x, N .dot(y) .add(x) .transpose() .sum(axis=1) ) h is strictly equivalent to import numpy as np x = np.array([[1,2],[3,4]]) y = np.array([[5,6],[7,8]]) z = np.dot(x, y) z = np.add(z, x) z = np.transpose(z) z = np.sum(z, axis=1) thing to notice is that with the `NumpyBuilder` we avoid the repetitive and needless passing and reassigment of the `z` variable, this removes a lot of noise from our code. """ _rtp = return_type_predicate return_type_predicate = (lambda x: _rtp) if inspect.isclass(_rtp) and issubclass(_rtp, Builder) else _rtp module_name = module_alias if module_alias else module.__name__ + '.' patch_members = _get_patch_members(module, blacklist_predicate=blacklist_predicate, whitelist_predicate=whitelist_predicate, getmembers_predicate=getmembers_predicate, admit_private=admit_private) if type(whitelist_predicate) is list and "convolution2d" in whitelist_predicate: import ipdb #ipdb.set_trace() for name, f in patch_members: wrapped = None if method_wrapper: g = method_wrapper(f) wrapped = f else: g = f cls.RegisterAt(n, g, module_name, wrapped=wrapped, _return_type=return_type_predicate(name), alias=method_name_modifier(name), explanation=explanation)
def Pipe(
self, *sequence, **kwargs)
Pipe
runs any phi.dsl.Expression
. Its highly inspired by Elixir's |> (pipe) operator.
Arguments
- *sequence: any variable amount of expressions. All expressions inside of
sequence
will be composed together usingphi.dsl.Expression.Seq
. - **kwargs:
Pipe
forwards allkwargs
toSeq
, visit its documentation for more info.
The expression
Pipe(*sequence, **kwargs)
is equivalent to
Seq(*sequence, **kwargs)(None)
Normally the first argument or Pipe
is a value, that is reinterpreted as a phi.dsl.Expression.Val
, therfore, the input None
is discarded.
Examples
from phi import P def add1(x): return x + 1 def mul3(x): return x * 3 x = P.Pipe( 1, #input add1, #1 + 1 == 2 mul3 #2 * 3 == 6 ) assert x == 6
The previous using lambdas to create the functions
from phi import P x = P.Pipe( 1, #input P + 1, #1 + 1 == 2 P * 3 #2 * 3 == 6 ) assert x == 6
Also see
def Pipe(self, *sequence, **kwargs): """ e` runs any `phi.dsl.Expression`. Its highly inspired by Elixir's [|> (pipe)](https://hexdocs.pm/elixir/Kernel.html#%7C%3E/2) operator. guments** *sequence**: any variable amount of expressions. All expressions inside of `sequence` will be composed together using `phi.dsl.Expression.Seq`. **kwargs**: `Pipe` forwards all `kwargs` to `phi.builder.Builder.Seq`, visit its documentation for more info. expression Pipe(*sequence, **kwargs) quivalent to Seq(*sequence, **kwargs)(None) ally the first argument or `Pipe` is a value, that is reinterpreted as a `phi.dsl.Expression.Val`, therfore, the input `None` is discarded. amples** from phi import P def add1(x): return x + 1 def mul3(x): return x * 3 x = P.Pipe( 1, #input add1, #1 + 1 == 2 mul3 #2 * 3 == 6 ) assert x == 6 previous using [lambdas](https://cgarciae.github.io/phi/lambdas.m.html) to create the functions from phi import P x = P.Pipe( 1, #input P + 1, #1 + 1 == 2 P * 3 #2 * 3 == 6 ) assert x == 6 so see** hi.builder.Builder.Seq` sl](https://cgarciae.github.io/phi/dsl.m.html) ompile](https://cgarciae.github.io/phi/dsl.m.html#phi.dsl.Compile) ambdas](https://cgarciae.github.io/phi/lambdas.m.html) """ state = kwargs.pop("refs", {}) return self.Seq(*sequence, **kwargs)(None, **state)
def ReadList(
self, *branches, **kwargs)
Same as phi.dsl.Expression.List
but any string argument x
is translated to Read(x)
.
def ReadList(self, *branches, **kwargs): """ as `phi.dsl.Expression.List` but any string argument `x` is translated to `Read(x)`. """ branches = map(lambda x: E.Read(x) if isinstance(x, str) else x, branches) return self.List(*branches, **kwargs)
def Register(
cls, *args, **kwargs)
@classmethod def Register(cls, *args, **kwargs): """ ister(...)` is a shortcut for `RegisterAt(1, ...)` so See** hi.builder.Builder.RegisterAt` hi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(1, *args, **kwargs)
def Register0(
cls, *args, **kwargs)
@classmethod def Register0(cls, *args, **kwargs): """ ister0(...)` is a shortcut for `RegisterAt(0, ...)` so See** hi.builder.Builder.RegisterAt` hi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(0, *args, **kwargs)
def Register2(
cls, *args, **kwargs)
@classmethod def Register2(cls, *args, **kwargs): """ ister2(...)` is a shortcut for `RegisterAt(2, ...)` so See** hi.builder.Builder.RegisterAt` hi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(2, *args, **kwargs)
def Register3(
cls, *args, **kwargs)
@classmethod def Register3(cls, *args, **kwargs): """ ister3(...)` is a shortcut for `RegisterAt(3, ...)` so See** hi.builder.Builder.RegisterAt` hi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(3, *args, **kwargs)
def Register4(
cls, *args, **kwargs)
@classmethod def Register4(cls, *args, **kwargs): """ ister4(...)` is a shortcut for `RegisterAt(4, ...)` so See** hi.builder.Builder.RegisterAt` hi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(4, *args, **kwargs)
def Register5(
cls, *args, **kwargs)
@classmethod def Register5(cls, *args, **kwargs): """ ister5(...)` is a shortcut for `RegisterAt(5, ...)` so See** hi.builder.Builder.RegisterAt` hi.builder.Builder.RegisterMethod` """ return cls.RegisterAt(5, *args, **kwargs)
def RegisterAt(
cls, *args, **kwargs)
RegisterAt
RegisterAt(n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None)
Most of the time you don't want to register an method as such, that is, you don't care about the self
builder object, instead you want to register a function that transforms the value being piped down the DSL. For this you can use RegisterAt
so e.g.
def some_fun(obj, arg1, arg2): # code @MyBuilder.RegisterMethod("my_lib.") def some_fun_wrapper(self, arg1, arg2): return self.ThenAt(1, some_fun, arg1, arg2)
can be written directly as
@MyBuilder.RegisterAt(1, "my_lib.") def some_fun(obj, arg1, arg2): # code
For this case you can just use Register
which is a shortcut for RegisterAt(1, ...)
@MyBuilder.Register("my_lib.") def some_fun(obj, arg1, arg2): # code
Also See
@classmethod def RegisterAt(cls, *args, **kwargs): """ gisterAt** RegisterAt(n, f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True, _return_type=None) of the time you don't want to register an method as such, that is, you don't care about the `self` builder object, instead you want to register a function that transforms the value being piped down the DSL. For this you can use `RegisterAt` so e.g. def some_fun(obj, arg1, arg2): # code @MyBuilder.RegisterMethod("my_lib.") def some_fun_wrapper(self, arg1, arg2): return self.ThenAt(1, some_fun, arg1, arg2) be written directly as @MyBuilder.RegisterAt(1, "my_lib.") def some_fun(obj, arg1, arg2): # code this case you can just use `Register` which is a shortcut for `RegisterAt(1, ...)` @MyBuilder.Register("my_lib.") def some_fun(obj, arg1, arg2): # code so See** hi.builder.Builder.RegisterMethod` """ unpack_error = True try: n, f, library_path = args unpack_error = False cls._RegisterAt(n, f, library_path, **kwargs) except: if not unpack_error: raise def register_decorator(f): n, library_path = args cls._RegisterAt(n, f, library_path, **kwargs) return f return register_decorator
def RegisterMethod(
cls, *args, **kwargs)
RegisterMethod
RegisterMethod(f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True)
classmethod
for registering functions as methods of this class.
Arguments
- f : the particular function being registered as a method
- library_path : library from where
f
comes from, unless you pass an empty string, put a period"."
at the end of the library name. alias=None
: alias for the name/method being registeredoriginal_name=None
: name of the original function, used for documentation purposes.doc=None
: complete documentation of the method being registeredwrapped=None
: if you are registering a function which wraps around another function, pass this other function throughwrapped
to get better documentation, this is specially useful is you register a bunch of functions in a for loop. Please include anexplanation
to tell how the actual function differs from the wrapped one.explanation=""
: especify any additional information for the documentation of the method being registered, you can use any of the following format tags within this string and they will be replace latter on:{original_name}
,{name}
,{fn_docs}
,{library_path}
,{builder_class}
.method_type=identity
: by default its applied but does nothing, you might also want to register functions asproperty
,classmethod
,staticmethod
explain=True
: decide whether or not to show any kind of explanation, its useful to set it toFalse
if you are using aRegister*
decorator and will only use the function as a registered method.
A main feature of phi
is that it enables you to integrate your library or even an existing library with the DSL. You can achieve three levels of integration
- Passing your functions to the DSL. This a very general machanism -since you could actually do everything with python lamdas- but in practice functions often receive multiple parameters.
- Creating partials with the
Then*
method family. Using this you could integrate any function, but it will add a lot of noise if you use heavily on it. - Registering functions as methods of a
Builder
derived class. This produces the most readable code and its the approach you should take if you want to create a Phi-based library or a helper class.
While point 3 is the most desirable it has a cost: you need to create your own Builder
-derived class. This is because SHOULD NOT register functions to existing builders e.g. the Builder
or PythonBuilder provided by phi because that would pollute the P
object. Instead you should create a custom class that derives from Builder
, PythonBuilder or another custom builder depending on your needs and register your functions to that class.
Examples
Say you have a function on a library called "my_lib"
def some_fun(obj, arg1, arg2): # code
You could use it with the dsl like this
from phi import P, Then P.Pipe( input, ... Then(some_fun, arg1, arg2) ... )
assuming the first parameter obj
is being piped down. However if you do this very often or you are creating a library, you are better off creating a custom class derived from Builder
or PythonBuilder
from phi import Builder #or PythonBuilder class MyBuilder(Builder): # or PythonBuilder pass
and registering your function as a method. The first way you could do this is by creating a wrapper function for some_fun
and registering it as a method
def some_fun_wrapper(self, arg1, arg2): return self.Then(some_fun, arg1, arg2) MyBuilder.RegisterMethod(some_fun_wrapper, "my_lib.", wrapped=some_fun)
Here we basically created a shortcut for the original expression Then(some_fun, arg1, arg2)
. You could also do this using a decorator
@MyBuilder.RegisterMethod("my_lib.", wrapped=some_fun) def some_fun_wrapper(self, arg1, arg2): return self.Then(some_fun, arg1, arg2)
However, this is such a common task that we've created the method Register
to avoid you from having to create the wrapper. With it you could register the function some_fun
directly as a method like this
MyBuilder.Register(some_fun, "my_lib.")
or by using a decorator over the original function definition
@MyBuilder.Register("my_lib.") def some_fun(obj, arg1, arg2): # code
Once done you've done any of the previous approaches you can create a custom global object e.g. M
and use it instead of/along with P
M = MyBuilder(lambda x: x) M.Pipe( input, ... M.some_fun(arg1, args) ... )
Argument position
Register
internally uses Then
, this is only useful if the object being piped is intended to be passed as the first argument of the function being registered, if this is not the case you could use Register2
, Register3
, ..., Register5
or RegisterAt
to set an arbitrary position, these functions will internally use Then2
, Then3
, ..., Then5
or ThenAt
respectively.
Wrapping functions
Sometimes you have an existing function that you would like to modify slightly so it plays nicely with the DSL, what you normally do is create a function that wraps around it and passes the arguments to it in a way that is convenient
import some_lib @MyBuilder.Register("some_lib.") def some_fun(a, n): return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified
When you do this -as a side effect- you loose the original documentation, to avoid this you can use the Registers wrapped
argument along with the explanation
argument to clarity the situation
import some_lib some_fun_explanation = "However, it differs in that `n` is automatically subtracted `1`" @MyBuilder.Register("some_lib.", wrapped=some_lib.some_fun, explanation=some_fun_explanation) def some_fun(a, n): return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified
Now the documentation for MyBuilder.some_fun
will be a little bit nicer since it includes the original documentation from some_lib.some_fun
. This behaviour is specially useful if you are wrapping an entire 3rd party library, you usually automate the process iterating over all the funcitions in a for loop. The PatchAt
method lets you register and entire module using a few lines of code, however, something you have to do thing more manually and do the iteration yourself.
See Also
@classmethod def RegisterMethod(cls, *args, **kwargs): """ gisterMethod** RegisterMethod(f, library_path, alias=None, original_name=None, doc=None, wrapped=None, explanation="", method_type=utils.identity, explain=True) ssmethod` for registering functions as methods of this class. guments** f** : the particular function being registered as a method library_path** : library from where `f` comes from, unless you pass an empty string, put a period `"."` at the end of the library name. lias=None` : alias for the name/method being registered riginal_name=None` : name of the original function, used for documentation purposes. oc=None` : complete documentation of the method being registered rapped=None` : if you are registering a function which wraps around another function, pass this other function through `wrapped` to get better documentation, this is specially useful is you register a bunch of functions in a for loop. Please include an `explanation` to tell how the actual function differs from the wrapped one. xplanation=""` : especify any additional information for the documentation of the method being registered, you can use any of the following format tags within this string and they will be replace latter on: `{original_name}`, `{name}`, `{fn_docs}`, `{library_path}`, `{builder_class}`. ethod_type=identity` : by default its applied but does nothing, you might also want to register functions as `property`, `classmethod`, `staticmethod` xplain=True` : decide whether or not to show any kind of explanation, its useful to set it to `False` if you are using a `Register*` decorator and will only use the function as a registered method. in feature of `phi` is that it enables you to integrate your library or even an existing library with the DSL. You can achieve three levels of integration assing your functions to the DSL. This a very general machanism -since you could actually do everything with python lamdas- but in practice functions often receive multiple parameters. reating partials with the `Then*` method family. Using this you could integrate any function, but it will add a lot of noise if you use heavily on it. egistering functions as methods of a `Builder` derived class. This produces the most readable code and its the approach you should take if you want to create a Phi-based library or a helper class. e point 3 is the most desirable it has a cost: you need to create your own `phi.builder.Builder`-derived class. This is because SHOULD NOT register functions to existing builders e.g. the `phi.builder.Builder` or [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) provided by phi because that would pollute the `P` object. Instead you should create a custom class that derives from `phi.builder.Builder`, [PythonBuilder](https://cgarciae.github.io/phi/builder.m.html#phi.python_builder.PythonBuilder) or another custom builder depending on your needs and register your functions to that class. amples** you have a function on a library called `"my_lib"` def some_fun(obj, arg1, arg2): # code could use it with the dsl like this from phi import P, Then P.Pipe( input, ... Then(some_fun, arg1, arg2) ... ) ming the first parameter `obj` is being piped down. However if you do this very often or you are creating a library, you are better off creating a custom class derived from `Builder` or `PythonBuilder` from phi import Builder #or PythonBuilder class MyBuilder(Builder): # or PythonBuilder pass registering your function as a method. The first way you could do this is by creating a wrapper function for `some_fun` and registering it as a method def some_fun_wrapper(self, arg1, arg2): return self.Then(some_fun, arg1, arg2) MyBuilder.RegisterMethod(some_fun_wrapper, "my_lib.", wrapped=some_fun) we basically created a shortcut for the original expression `Then(some_fun, arg1, arg2)`. You could also do this using a decorator @MyBuilder.RegisterMethod("my_lib.", wrapped=some_fun) def some_fun_wrapper(self, arg1, arg2): return self.Then(some_fun, arg1, arg2) ver, this is such a common task that we've created the method `Register` to avoid you from having to create the wrapper. With it you could register the function `some_fun` directly as a method like this MyBuilder.Register(some_fun, "my_lib.") y using a decorator over the original function definition @MyBuilder.Register("my_lib.") def some_fun(obj, arg1, arg2): # code done you've done any of the previous approaches you can create a custom global object e.g. `M` and use it instead of/along with `P` M = MyBuilder(lambda x: x) M.Pipe( input, ... M.some_fun(arg1, args) ... ) gument position** .builder.Builder.Register` internally uses `phi.builder.Builder.Then`, this is only useful if the object being piped is intended to be passed as the first argument of the function being registered, if this is not the case you could use `phi.builder.Builder.Register2`, `phi.builder.Builder.Register3`, ..., `phi.builder.Builder.Register5` or `phi.builder.Builder.RegisterAt` to set an arbitrary position, these functions will internally use `phi.builder.Builder.Then2`, `phi.builder.Builder.Then3`, ..., `phi.builder.Builder.Then5` or `phi.builder.Builder.ThenAt` respectively. apping functions** times you have an existing function that you would like to modify slightly so it plays nicely with the DSL, what you normally do is create a function that wraps around it and passes the arguments to it in a way that is convenient import some_lib @MyBuilder.Register("some_lib.") def some_fun(a, n): return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified you do this -as a side effect- you loose the original documentation, to avoid this you can use the Registers `wrapped` argument along with the `explanation` argument to clarity the situation import some_lib some_fun_explanation = "However, it differs in that `n` is automatically subtracted `1`" @MyBuilder.Register("some_lib.", wrapped=some_lib.some_fun, explanation=some_fun_explanation) def some_fun(a, n): return some_lib.some_fun(a, n - 1) # forward the args, n slightly modified the documentation for `MyBuilder.some_fun` will be a little bit nicer since it includes the original documentation from `some_lib.some_fun`. This behaviour is specially useful if you are wrapping an entire 3rd party library, you usually automate the process iterating over all the funcitions in a for loop. The `phi.builder.Builder.PatchAt` method lets you register and entire module using a few lines of code, however, something you have to do thing more manually and do the iteration yourself. e Also** hi.builder.Builder.PatchAt` hi.builder.Builder.RegisterAt` """ unpack_error = True try: f, library_path = args unpack_error = False cls._RegisterMethod(f, library_path, **kwargs) except: if not unpack_error: raise def register_decorator(f): library_path, = args cls._RegisterMethod(f, library_path, **kwargs) return f return register_decorator
def Seq(
self, *sequence, **kwargs)
Seq
is used to express function composition. The expression
Seq(f, g)
be equivalent to
lambda x: g(f(x))
As you see, its a little different from the mathematical definition. Excecution order flow from left to right, this makes reading and reasoning about code way more easy. This bahaviour is based upon the |>
(pipe) operator found in languages like F#, Elixir and Elm. You can pack as many expressions as you like and they will be applied in order to the data that is passed through them when compiled an excecuted.
In general, the following rules apply for Seq:
General Sequence
Seq(f0, f1, ..., fn-1, fn)
is equivalent to
lambda x: fn(fn-1(...(f1(f0(x)))))
Single Function
Seq(f)
is equivalent to
f
Identity
The empty Seq
Seq()
is equivalent to
lambda x: x
Examples
from phi import P, Seq f = Seq( P * 2, P + 1, P ** 2 ) assert f(1) == 9 # ((1 * 2) + 1) ** 2
The previous example using P.Pipe
from phi import P assert 9 == P.Pipe( 1, P * 2, #1 * 2 == 2 P + 1, #2 + 1 == 3 P ** 2 #3 ** 2 == 9 )
def Seq(self, *sequence, **kwargs): """ ` is used to express function composition. The expression Seq(f, g) quivalent to lambda x: g(f(x)) ou see, its a little different from the mathematical definition. Excecution order flow from left to right, this makes reading and reasoning about code way more easy. This bahaviour is based upon the `|>` (pipe) operator found in languages like F#, Elixir and Elm. You can pack as many expressions as you like and they will be applied in order to the data that is passed through them when compiled an excecuted. eneral, the following rules apply for Seq: neral Sequence** Seq(f0, f1, ..., fn-1, fn) quivalent to lambda x: fn(fn-1(...(f1(f0(x))))) ngle Function** Seq(f) quivalent to f entity** empty Seq Seq() quivalent to lambda x: x Examples from phi import P, Seq f = Seq( P * 2, P + 1, P ** 2 ) assert f(1) == 9 # ((1 * 2) + 1) ** 2 previous example using `P.Pipe` from phi import P assert 9 == P.Pipe( 1, P * 2, #1 * 2 == 2 P + 1, #2 + 1 == 3 P ** 2 #3 ** 2 == 9 ) """ fs = [ _parse(elem)._f for elem in sequence ] def g(x, state): return functools.reduce(lambda args, f: f(*args), fs, (x, state)) return self.__then__(g, **kwargs)
def Set(
self, *expressions, **kwargs)
def Set(self, *expressions, **kwargs): return self.List(*expressions) >> set
def Then(
self, f, *args, **kwargs)
Then(f, ...)
is equivalent to ThenAt(1, f, ...)
. Checkout ThenAt
for more information.
def Then(self, f, *args, **kwargs): """ n(f, ...)` is equivalent to `ThenAt(1, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information. """ return self.ThenAt(1, f, *args, **kwargs)
def Then0(
self, f, *args, **kwargs)
Then0(f, ...)
is equivalent to ThenAt(0, f, ...)
. Checkout ThenAt
for more information.
def Then0(self, f, *args, **kwargs): """ n0(f, ...)` is equivalent to `ThenAt(0, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information. """ return self.ThenAt(0, f, *args, **kwargs)
def Then1(
self, f, *args, **kwargs)
Then(f, ...)
is equivalent to ThenAt(1, f, ...)
. Checkout ThenAt
for more information.
def Then(self, f, *args, **kwargs): """ n(f, ...)` is equivalent to `ThenAt(1, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information. """ return self.ThenAt(1, f, *args, **kwargs)
def Then2(
self, f, arg1, *args, **kwargs)
Then2(f, ...)
is equivalent to ThenAt(2, f, ...)
. Checkout ThenAt
for more information.
def Then2(self, f, arg1, *args, **kwargs): """ n2(f, ...)` is equivalent to `ThenAt(2, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information. """ args = (arg1,) + args return self.ThenAt(2, f, *args, **kwargs)
def Then3(
self, f, arg1, arg2, *args, **kwargs)
Then3(f, ...)
is equivalent to ThenAt(3, f, ...)
. Checkout ThenAt
for more information.
def Then3(self, f, arg1, arg2, *args, **kwargs): """ n3(f, ...)` is equivalent to `ThenAt(3, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information. """ args = (arg1, arg2) + args return self.ThenAt(3, f, *args, **kwargs)
def Then4(
self, f, arg1, arg2, arg3, *args, **kwargs)
Then4(f, ...)
is equivalent to ThenAt(4, f, ...)
. Checkout ThenAt
for more information.
def Then4(self, f, arg1, arg2, arg3, *args, **kwargs): """ n4(f, ...)` is equivalent to `ThenAt(4, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information. """ args = (arg1, arg2, arg3) + args return self.ThenAt(4, f, *args, **kwargs)
def Then5(
self, f, arg1, arg2, arg3, arg4, *args, **kwargs)
Then5(f, ...)
is equivalent to ThenAt(5, f, ...)
. Checkout ThenAt
for more information.
def Then5(self, f, arg1, arg2, arg3, arg4, *args, **kwargs): """ n5(f, ...)` is equivalent to `ThenAt(5, f, ...)`. Checkout `phi.builder.Builder.ThenAt` for more information. """ args = (arg1, arg2, arg3, arg4) + args return self.ThenAt(5, f, *args, **kwargs)
def ThenAt(
self, n, f, *_args, **kwargs)
ThenAt
enables you to create a partially apply many arguments to a function, the returned partial expects a single arguments which will be applied at the n
th position of the original function.
Arguments
- n: position at which the created partial will apply its awaited argument on the original function.
- f: function which the partial will be created.
- _args & kwargs: all
*_args
and**kwargs
will be passed to the functionf
. _return_type = None
: type of the returnedbuilder
, ifNone
it will return the same type of the currentbuilder
. This special kwarg will NOT be passed tof
.
You can think of n
as the position that the value being piped down will pass through the f
. Say you have the following expression
D == fun(A, B, C)
all the following are equivalent
from phi import P, Pipe, ThenAt D == Pipe(A, ThenAt(1, fun, B, C)) D == Pipe(B, ThenAt(2, fun, A, C)) D == Pipe(C, ThenAt(3, fun, A, B))
you could also use the shortcuts Then
, Then2
,..., Then5
, which are more readable
from phi import P, Pipe D == Pipe(A, P.Then(fun, B, C)) D == Pipe(B, P.Then2(fun, A, C)) D == Pipe(C, P.Then3(fun, A, B))
There is a special case not discussed above: n = 0
. When this happens only the arguments given will be applied to f
, this method it will return a partial that expects a single argument but completely ignores it
from phi import P D == Pipe(None, P.ThenAt(0, fun, A, B, C)) D == Pipe(None, P.Then0(fun, A, B, C))
Examples
Max of 6 and the argument:
from phi import P assert 6 == P.Pipe( 2, P.Then(max, 6) )
Previous is equivalent to
assert 6 == max(2, 6)
Open a file in read mode ('r'
)
from phi import P f = P.Pipe( "file.txt", P.Then(open, 'r') )
Previous is equivalent to
f = open("file.txt", 'r')
Split a string by whitespace and then get the length of each word
from phi import P assert [5, 5, 5] == P.Pipe( "Again hello world", P.Then(str.split, ' ') .Then2(map, len) )
Previous is equivalent to
x = "Again hello world" x = str.split(x, ' ') x = map(len, x) assert [5, 5, 5] == x
As you see, Then2
was very useful because map
accepts and iterable
as its 2nd
parameter. You can rewrite the previous using the PythonBuilder and the Obj
object
from phi import P, Obj assert [5, 5, 5] == P.Pipe( "Again hello world", Obj.split(' '), P.map(len) )
Also see
def ThenAt(self, n, f, *_args, **kwargs): """ nAt` enables you to create a partially apply many arguments to a function, the returned partial expects a single arguments which will be applied at the `n`th position of the original function. guments** n**: position at which the created partial will apply its awaited argument on the original function. f**: function which the partial will be created. _args & kwargs**: all `*_args` and `**kwargs` will be passed to the function `f`. return_type = None`: type of the returned `builder`, if `None` it will return the same type of the current `builder`. This special kwarg will NOT be passed to `f`. can think of `n` as the position that the value being piped down will pass through the `f`. Say you have the following expression D == fun(A, B, C) the following are equivalent from phi import P, Pipe, ThenAt D == Pipe(A, ThenAt(1, fun, B, C)) D == Pipe(B, ThenAt(2, fun, A, C)) D == Pipe(C, ThenAt(3, fun, A, B)) could also use the shortcuts `Then`, `Then2`,..., `Then5`, which are more readable from phi import P, Pipe D == Pipe(A, P.Then(fun, B, C)) D == Pipe(B, P.Then2(fun, A, C)) D == Pipe(C, P.Then3(fun, A, B)) e is a special case not discussed above: `n = 0`. When this happens only the arguments given will be applied to `f`, this method it will return a partial that expects a single argument but completely ignores it from phi import P D == Pipe(None, P.ThenAt(0, fun, A, B, C)) D == Pipe(None, P.Then0(fun, A, B, C)) amples** of 6 and the argument: from phi import P assert 6 == P.Pipe( 2, P.Then(max, 6) ) ious is equivalent to assert 6 == max(2, 6) a file in read mode (`'r'`) from phi import P f = P.Pipe( "file.txt", P.Then(open, 'r') ) ious is equivalent to f = open("file.txt", 'r') t a string by whitespace and then get the length of each word from phi import P assert [5, 5, 5] == P.Pipe( "Again hello world", P.Then(str.split, ' ') .Then2(map, len) ) ious is equivalent to x = "Again hello world" x = str.split(x, ' ') x = map(len, x) assert [5, 5, 5] == x ou see, `Then2` was very useful because `map` accepts and `iterable` as its `2nd` parameter. You can rewrite the previous using the [PythonBuilder](https://cgarciae.github.io/phi/python_builder.m.html) and the `phi.builder.Builder.Obj` object from phi import P, Obj assert [5, 5, 5] == P.Pipe( "Again hello world", Obj.split(' '), P.map(len) ) so see** hi.builder.Builder.Obj` ythonBuilder](https://cgarciae.github.io/phi/python_builder.m.html) hi.builder.Builder.RegisterAt` """ _return_type = None n_args = n - 1 if '_return_type' in kwargs: _return_type = kwargs['_return_type'] del kwargs['_return_type'] @utils.lift def g(x): new_args = _args[0:n_args] + (x,) + _args[n_args:] if n_args >= 0 else _args return f(*new_args, **kwargs) return self.__then__(g, _return_type=_return_type)
def Tuple(
self, *expressions, **kwargs)
def Tuple(self, *expressions, **kwargs): return self.List(*expressions) >> tuple
def Val(
self, val, **kwargs)
The expression
Val(a)
is equivalent to the constant function
lambda x: a
All expression in this module interprete values that are not functions as constant functions using Val
, for example
Seq(1, P + 1)
is equivalent to
Seq(Val(1), P + 1)
The previous expression as a whole is a constant function since it will return 2
no matter what input you give it.
def Val(self, val, **kwargs): """ expression Val(a) quivalent to the constant function lambda x: a expression in this module interprete values that are not functions as constant functions using `Val`, for example Seq(1, P + 1) quivalent to Seq(Val(1), P + 1) previous expression as a whole is a constant function since it will return `2` no matter what input you give it. """ f = utils.lift(lambda z: val) return self.__then__(f, **kwargs)
def With(
self, context_manager, *body, **kwargs)
With
def With(context_manager, *body):
Arguments
- context_manager: a context manager object or valid expression from the DSL that returns a context manager.
- *body: any valid expression of the DSL to be evaluated inside the context.
*body
is interpreted as a tuple so all expression contained are composed.
As with normal python programs you sometimes might want to create a context for a block of code. You normally give a context manager to the with statemente, in Phi you use P.With
or phi.With
Context
Python's with
statemente returns a context object through as
keyword, in the DSL this object can be obtained using the P.Context
method or the phi.Context
function.
Examples
from phi import P, Obj, Context, With, Pipe text = Pipe( "text.txt", With( open, Context, Obj.read() ) )
The previous is equivalent to
with open("text.txt") as f: text = f.read()
def With(self, context_manager, *body, **kwargs): """ th** def With(context_manager, *body): guments** context_manager**: a [context manager](https://docs.python.org/2/reference/datamodel.html#context-managers) object or valid expression from the DSL that returns a context manager. *body**: any valid expression of the DSL to be evaluated inside the context. `*body` is interpreted as a tuple so all expression contained are composed. ith normal python programs you sometimes might want to create a context for a block of code. You normally give a [context manager](https://docs.python.org/2/reference/datamodel.html#context-managers) to the [with](https://docs.python.org/2/reference/compound_stmts.html#the-with-statement) statemente, in Phi you use `P.With` or `phi.With` ntext** on's `with` statemente returns a context object through `as` keyword, in the DSL this object can be obtained using the `P.Context` method or the `phi.Context` function. Examples from phi import P, Obj, Context, With, Pipe text = Pipe( "text.txt", With( open, Context, Obj.read() ) ) previous is equivalent to with open("text.txt") as f: text = f.read() """ context_f = _parse(context_manager)._f body_f = E.Seq(*body)._f def g(x, state): context, state = context_f(x, state) with context as scope: with _WithContextManager(scope): return body_f(x, state) return self.__then__(g, **kwargs)
def Write(
self, *state_args, **state_dict)
See phi.dsl.Expression.Read
def Write(self, *state_args, **state_dict): """See `phi.dsl.Expression.Read`""" if len(state_dict) + len(state_args) < 1: raise Exception("Please include at-least 1 state variable, got {0} and {1}".format(state_args, state_dict)) if len(state_dict) > 1: raise Exception("Please include at-most 1 keyword argument expression, got {0}".format(state_dict)) if len(state_dict) > 0: state_key = next(iter(state_dict.keys())) write_expr = state_dict[state_key] state_args += (state_key,) expr = self >> write_expr else: expr = self def g(x, state): update = { key: x for key in state_args } state = utils.merge(state, update) #side effect for convenience _StateContextManager.REFS.update(state) return x, state return expr.__then__(g)