Metadata-Version: 1.1
Name: yaql
Version: 0.2.6
Summary: YAQL - Yet Another Query Language
Home-page: http://www.openstack.org/
Author: OpenStack
Author-email: openstack-dev@lists.openstack.org
License: UNKNOWN
Description: YAQL - Yet Another Query Language
        =================================
        
        At the beginning of millennium the growing trend towards data formats standardization and application integrability made
        XML extremely popular. XML became lingua franca of the data. Applications tended to process lots of XML files ranging
        from small config files to very large datasets. As these data often had a complex structure with many levels of
        nestedness it is quickly became obvious that there is a need for specially crafted domain specific languages to query
        these data sets. This  is how XPath and later XQL were born.
        
        With later popularization of REST services and Web 2.0 JSON started to take XML’s place. JSON’s main advantage (besides
        being simpler than XML) is that is closely reassembles data structures found in most programming languages (arrays,
        dictionaries, scalars) making it very convenient for data serialization. As JSON lacked all the brilliant XML-related
        technologies like XSLT, XML Schema, XPath etc. various attempts to develop similar languages for JSON were made. One of
        those efforts was JSONPath library developed in 2007 by Stefan Gössner. Initial implementation was for PHP and
        JavaScript languages, but later on ports to other languages including Python were written.
        
        JSONPath allows navigation and querying, well, JSONs.
        Suppose we have JSON as in following:
        
            {
              "customers": [
              {
                "customer_id": 1,
                "name": "John",
                "orders": [{
                  "order_id": 1,
                  "item": "Guitar",
                 "quantity": 1
                }]
              },{
               "customer_id": 2,
               "name": "Paul",
               "orders": [ {
                  "order_id": 2,
                  "item": "Banjo",
                  "quantity": 2
                 },{
                 "order_id": 3,
                 "item": "Piano",
                 "quantity": 1
                }]
              }
             ]
            }
        
        
        then
        
        `jsonpath(data, "$.customers[0].name") -> [‘John’]`
        `jsonpath(data, "$.customers[*].orders[*].order_id") -> [1, 2, 3]`
        
        But what if we need, for example to find order having ID = 2? Here is how it done in JSONPath:
        
        `jsonpath(data, "$.customers[*].orders[?(@.order_id == 2)") ->  [{'order_id': 2, 'item': 'Banjo', 'quantity': 2}]`
        
        The construct `[?(expression)]` allows to filter items using any Python expression in our case. `@` character is
        replaced with current value and then the whole expression is evaluated. Evaluation of arbitrary Python expression
        requires using `eval()` function unless one wants to develop his own complete parser and interpreter of Python
        programming language. Needless to say that `eval()` is a great security breach. If JSONPath expressions are used to
        simplify program logic it would not be a big deal, but what if JSONPath is written by program users?
        
        JSONPath expression is just a plain string. There is no such concept as parameter. That is if one want to find order
        having ID = some variable value he has to dynamically construct expression string using string formatting or
        concatenation. And again that is might be okay for internal usage but would became difficult for external usage and also
        open the doors for injection attacks (remember SQL injection?)
        
        Another limitation of JSONPath is JSON itself. Technically speaking JSONPath operates not on the JSON itself (i.e. text
        representation) but on a JSON-like object model that is mixture of arrays, dictionaries and scalar values. But what is
        one want to query object model consisting of custom objects? What if some parts of this model are dynamically computed?
        Or the model is a graph rather than a tree?
        
        It seems like JSONPath is good enough to use in Python code when you can `eval()` things and have many helper function
        to work with data besides JSONPath capabilities but is not enough for external use when you need to have sufficient
        power to query model without manual coding and have it still secure.
        This is why we designed YAQL. YAQL follows the JSONPath ideas and has very similar syntax but offers much more for data
        querying.
        
        Expressions are quite similar to JSONPath. Here is how examples above can be translated to YAQL:
        
        `$.customers[0].name -> $.customers[0].name (no change)`
        `$.customers[*].orders[*].order_id -> $.customers.orders.order_id`
        
        the main addition to JSONPath is functions and operators. Consider the following YAQL expressions:
        
        `$.customers.orders[$.quantity > 0].quantity.sum() -> 4`
        `$.customers.orders.select($.quantity * $.quantity).sum() -> 6`
        `$.customers.orders.order_id.orderDesc($) -> [3, 2, 1]`
        `$.customers.orders.order_id.orderDesc($).take(2) -> [3, 2]`
        `$.customers.orders.order_id.orderDesc($).first() -> 3`
        
        Does it mean that YAQL has large built-in function and operator library?. Yes, YAQL library has a out of the box large
        set of commonly used functions. But they are not built-in. All the functions and operators (which are also function:
        `a + b = operator_+(a, b)` etc) are user-supplied. User is free to add other functions that could be used in expressions
        and to remove standard ones.
        
        JSONPath library needs 2 arguments - input JSON data and an a expression. YAQL library requires third
        parameter - context.
        
        Context is a repository of functions and variables that can be used in expressions. So all the functions above are just
        ordinary Python functions that are registered in Context object. But because they all need to be registered in Context
        user can always customize them, add his own model-specific ones and have full control over the expression evaluation.
        
        
Platform: UNKNOWN
Classifier: Environment :: OpenStack
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.3
