blog

Data pipeline

Luigi  conda install -c conda-forge luigi=2.6.1

Machine learning

xgboost conda install -c creditx xgboost=0.64

Test

Pytest conda install -c conda-forge pytest=3.1.1

Dash board

Superset(based on Flask) http://airbnb.io/superset/installation.html

Search engine

Elasticsearch conda install -c conda-forge elasticsearch=5.3.0
Elasticsearch download link https://www.elastic.co/downloads/elasticsearch 
Launch elasticsearch $ path_to_elasticsearch\bin> elasticsearch (default port: 9200)
Elasticsearch toolbox (Chrome extension): view data without Kibana https://chrome.google.com/webstore/detail/elasticsearch-toolbox/focdbmjgdonlpdknobfghplhmafpgfbp

Database

Neo4j conda install -c tcallen07 neo4j-driver=1.1.0b3
Neo4j link https://neo4j.com/download/community-edition/
Launch Neo4j $ path_to_Neo4j Community Edition\bin neo4j-ce.exe (default port:7474)

PostgreSQL pip install psycopg2
PostgreSQL link https://www.enterprisedb.com/downloads/postgres-postgresql-downloads

MySQL https://sourceforge.net/projects/mysql-python/
MySQL pip install MySQL-python
Example
 

import MySQLdb
try:
    # Open database connection
    db = MySQLdb.connect(host = "host_ip"
                        , user = "user_name"
                        , passwd = "password"
                        , db = "database_name"
                        , charset = 'utf8')
    # prepare a cursor object using cursor() method
    cursor = db.cursor()
    try:     
        cursor.execute("""SELECT * FROM {table_name};""")
        for row in cursor.fetchall():
            print row

    except MySQLdb.Error as e:
        print "Error %d: %s" % (e.args[0], e.args[1])
        db.rollback()
except MySQLdb.Error as e:
    print "Error %d: %s" % (e.args[0], e.args[1])
finally:
    # disconnect from server
    cursor.close()
    db.close()

Data manipulation

conda install -c conda-forge numpy