Metadata-Version: 1.2
Name: monasca_transform
Version: 1.0.1.dev2
Summary: Data Aggregation and Transformation component for Monasca
Home-page: https://wiki.openstack.org/wiki/Monasca/Transform
Author: OpenStack
Author-email: openstack-discuss@lists.openstack.org
License: UNKNOWN
Description: Team and repository tags
        ========================
        
        .. image:: https://governance.openstack.org/tc/badges/monasca-transform.svg
            :target: https://governance.openstack.org/tc/reference/tags/index.html
        
        -  `Monasca Transform`_
        
           -  `Use Cases handled by Monasca Transform`_
           -  `Operation`_
           -  `Architecture`_
           -  `To set up the development environment`_
           -  `Generic aggregation components`_
           -  `Create a new aggregation pipeline example`_
           -  `Original proposal and blueprint`_
        
        Monasca Transform
        =================
        
        monasca-transform is a data driven aggregation engine which collects,
        groups and aggregates existing individual Monasca metrics according to
        business requirements and publishes new transformed (derived) metrics to
        the Monasca Kafka queue.
        
        -  Since the new transformed metrics are published as any other metric
           in Monasca, alarms can be set and triggered on the transformed
           metric.
        
        -  Monasca Transform uses `Apache Spark`_ to aggregate data. `Apache
           Spark`_ is a highly scalable, fast, in-memory, fault tolerant and
           parallel data processing framework. All monasca-transform components
           are implemented in Python and use Spark’s `PySpark Python API`_ to
           interact with Spark.
        
        -  Monasca Transform does transformation and aggregation of incoming
           metrics in two phases.
        
           -  In the first phase spark streaming application is set to retrieve
              in data from kafka at a configurable *stream interval* (default
              *stream_inteval* is 10 minutes) and write the data aggregated for
              *stream interval* to *pre_hourly_metrics* topic in kafka.
        
           -  In the second phase, which is kicked off every hour, all metrics
              in *metrics_pre_hourly* topic in Kafka are aggregated again, this
              time over a larger interval of an hour. These hourly aggregated
              metrics published to *metrics* topic in kafka.
        
        Use Cases handled by Monasca Transform
        --------------------------------------
        
        Please refer to **Problem Description** section on the
        `Monasca/Transform wiki`_
        
        Operation
        ---------
        
        Please refer to **How Monasca Transform Operates** section on the
        `Monasca/Transform wiki`_
        
        Architecture
        ------------
        
        Please refer to **Architecture** and **Logical processing data flow**
        sections on the `Monasca/Transform wiki`_
        
        To set up the development environment
        -------------------------------------
        
        The monasca-transform uses `DevStack`_ as a common dev environment. See
        the `README.md`_ in the devstack directory for details on how to include
        monasca-transform in a DevStack deployment.
        
        Generic aggregation components
        ------------------------------
        
        Monasca Transform uses a set of generic aggregation components which can
        be assembled in to an aggregation pipeline.
        
        Please refer to the
        `generic-aggregation-components`_
        document for information on list of generic aggregation components
        available.
        
        Create a new aggregation pipeline example
        -----------------------------------------
        
        Generic aggregation components make it easy to build new aggregation
        pipelines for different Monasca metrics.
        
        This create a `new aggregation pipeline`_ example shows how to create
        *pre_transform_specs* and *transform_specs* to create an aggregation
        pipeline for a new set of Monasca metrics, while leveraging existing set
        of generic aggregation components.
        
        Original proposal and blueprint
        -------------------------------
        
        Original proposal: `Monasca/Transform-proposal`_
        
        Blueprint: `monasca-transform blueprint`_
        
        .. _Apache Spark: https://spark.apache.org
        .. _generic-aggregation-components: docs/generic-aggregation-components.md
        .. _PySpark Python API: https://spark.apache.org/docs/latest/api/python/index.html
        .. _Monasca/Transform wiki: https://wiki.openstack.org/wiki/Monasca/Transform
        .. _DevStack: https://docs.openstack.org/devstack/latest/
        .. _README.md: devstack/README.md
        .. _new aggregation pipeline: docs/create-new-aggregation-pipeline.md
        .. _Monasca/Transform-proposal: https://wiki.openstack.org/wiki/Monasca/Transform-proposal
        .. _monasca-transform blueprint: https://blueprints.launchpad.net/monasca/+spec/monasca-transform
        
        
Platform: UNKNOWN
Classifier: Environment :: OpenStack
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Requires-Python: >=3.6
