Role Detail

chilcano.apache-nifi

An Ansible Role that installs and runs Apache NiFi.
Downloads
77
Type Ansible
Minimum Ansible Version 2.0
Installation $ ansible-galaxy install chilcano.apache-nifi
Tags
bigdata
dataflow
datamediation
datarouting
eip
iot
security
system
Last Commit
Last Imported
OS Platforms
Platform Version
Debian etch
Debian jessie
Debian lenny
Debian sid
Debian squeeze
Debian stretch
Debian wheezy
EL 6
EL 7
Fedora 16
Fedora 17
Fedora 18
Fedora 19
Fedora 20
Fedora 21
Fedora 22
Fedora 23
Fedora 24
Fedora 25
Fedora 26
Ubuntu artful
Ubuntu lucid
Ubuntu maverick
Ubuntu natty
Ubuntu oneiric
Ubuntu precise
Ubuntu quantal
Ubuntu raring
Ubuntu saucy
Ubuntu trusty
Ubuntu utopic
Ubuntu vivid
Ubuntu wily
Ubuntu xenial
Ubuntu yakkety
Ubuntu zesty
Last 10 Imports
Completed Status
SUCCESS
SUCCESS
SUCCESS

Ansible Role: apache-nifi

An Ansible Role that installs, configures and runs Apache NiFi. This Role provides the following features:

  • Installs Apache NiFi.
  • Configures Apache NiFi in 3 modes:
    • http: NiFi Standard configuration, no authentication and running over HTTP.
    • https: Configure and run NiFi over SSL/TLS with Authentication based on Client Certificate.
    • tls_toolkit: Pickup the Key-pair, Digital Certificates, Java Key Store and configuration files generated by Apache NiFi to configure and run NiFi securely.
  • Runs the Apache NiFi as a systemd service.

Requirements

  • Java.

Role Variables

Default variables are in defaults/main.yml.

Dependencies

This Ansible Role has not dependencies, although Apache NiFi requires Java. To install Java I will use the (geerlingguy.java)[https://github.com/geerlingguy/ansible-role-java] role but with a few changes to install Oracle Java 8 in Debian. You have these changes available from my fork at (chilcano.java / branch oracle-java-debian)[https://github.com/chilcano/ansible-role-java/tree/oracle-java-debian]. Just clone that branch.

But if you are going to use the Apache NiFi in Ubuntu or CentOS, you will not need the above changes. Just download the geerlingguy.java as below, or install manually Java 7 or 8 in your box:

$ sudo ansible-galaxy install geerlingguy.java

Example Playbook

- hosts: nf1
  become: yes
  vars_files:
    - vars.yml
  roles:
    - role: ../../playbooks/roles/ansible-role-java
      java_packages:
        - oracle-java8-installer
        - ca-certificates
        - oracle-java8-set-default
      java_cleanup: false
      java_home: "/usr/lib/jvm/java-8-oracle"

    - role: chilcano.apache-nifi
      nifi:
        version: "1.2.0"
        packaging_bin: "tar.gz"
        packaging_src: "zip"
        action:
          clean:
            installer: false
            installation: false
            dependencies: false
          install: true
          run: true
        deployment:
          mode: "http"
          tls_toolkit:
            shared_dir_local: "/Users/Chilcano/1github-repo/binaries"
            hostname: ""
            dir_repo: ""
        download:
          http_uri: "http://mirror.ox.ac.uk/sites/rsync.apache.org"
      nifi_conf_authorizers:
        initial_admin_identity: ""
        nf_hostname_pattern: ""
        nf_domain: ""
        nf_dn_suffix: ""
      nifi_properties_j2:
        nifi.remote.input.host: ""
        nifi.remote.input.secure: false
        nifi.remote.input.socket.port:
        nifi.web.http.host: ""
        nifi.web.http.port: 8080
        nifi.web.https.host: ""
        nifi.web.https.port:
        nifi.security.keystore: ""
        nifi.security.keystoreType: ""
        nifi.security.keystorePasswd: ""
        nifi.security.keyPasswd: ""
        nifi.security.truststore: ""
        nifi.security.truststoreType: ""
        nifi.security.truststorePasswd: ""
        nifi.security.user.authorizer: ""
        nifi.security.user.login.identity.provider: ""
        nifi.cluster.protocol.is.secure: false
        nifi.cluster.node.address: ""
        nifi.cluster.node.protocol.port:
      nifi_sync_dir_local: "/Users/Chilcano/1github-repo/binaries"

- hosts: nf2
  become: yes
  vars_files:
    - vars.yml
  roles:
    - role: ../../playbooks/roles/ansible-role-java
      java_packages:
        - oracle-java8-installer
        - ca-certificates
        - oracle-java8-set-default
      java_cleanup: false
      java_home: "/usr/lib/jvm/java-8-oracle"

    - role: chilcano.apache-nifi
      nifi:
        action:
          clean:
            installer: false
            installation: false
            dependencies: false
          install: true
          run: true
        deployment:
          mode: "https"
          tls_toolkit:
            shared_dir_local: "/Users/Chilcano/1github-repo/binaries"
            hostname: "nftk1"
            dir_repo: "nifi-toolkit_repo"
        download:
          http_uri: "http://mirror.ox.ac.uk/sites/rsync.apache.org"
      nifi_conf_authorizers:
        initial_admin_identity: "{{ _nftk_client_cert_dn }}"
        nf_hostname_pattern: "{{ _nftk_hostname }}"
        nf_domain: "{{ _nftk_domain }}"
        nf_dn_suffix: "{{ _nftk_dn_suffix }}"
      nifi_properties_j2:
        nifi.remote.input.host: ""
        nifi.remote.input.secure: true
        nifi.remote.input.socket.port: 10443
        nifi.web.http.host: ""
        nifi.web.http.port:
        nifi.web.https.host: ""
        nifi.web.https.port: 9443
        nifi.security.keystore: ./conf/keystore.jks
        nifi.security.keystoreType: jks
        nifi.security.keystorePasswd: "{{ _nftk_run_keystorepasswd }}"
        nifi.security.keyPasswd: "{{ _nftk_run_keypasswd }}"
        nifi.security.truststore: ./conf/truststore.jks
        nifi.security.truststoreType: jks
        nifi.security.truststorePasswd: "{{ _nftk_run_truststorepasswd }}"
        nifi.security.user.authorizer: "file-provider"
        nifi.security.user.login.identity.provider: ""
        nifi.cluster.protocol.is.secure: true
        nifi.cluster.node.address: ""
        nifi.cluster.node.protocol.port: 11443
      nifi_sync_dir_local: "/Users/Chilcano/1github-repo/binaries"

- hosts: nf3
  become: yes
  vars_files:
    - vars.yml
  roles:
    - role: ../../playbooks/roles/ansible-role-java
      java_packages:
        - oracle-java8-installer
        - ca-certificates
        - oracle-java8-set-default
      java_cleanup: false
      java_home: "/usr/lib/jvm/java-8-oracle"

    - role: chilcano.apache-nifi
      nifi:
        action:
          clean:
            installer: false
            installation: false
            dependencies: false
          install: true
          run: true
        deployment:
          mode: "tls_toolkit"
          tls_toolkit:
            shared_dir_local: "/Users/Chilcano/1github-repo/binaries"
            hostname: "nftk1"
            dir_repo: "nifi-toolkit_repo"
        download:
          http_uri: "http://mirror.ox.ac.uk/sites/rsync.apache.org"
      nifi_conf_authorizers:
        initial_admin_identity: "{{ _nftk_client_cert_dn }}"
        nf_hostname_pattern: "{{ _nftk_hostname }}"
        nf_domain: "{{ _nftk_domain }}"
        nf_dn_suffix: "{{ _nftk_dn_suffix }}"
      nifi_properties_j2:
        nifi.remote.input.host: ""
        nifi.remote.input.secure: true
        nifi.remote.input.socket.port: 10443
        nifi.web.http.host: ""
        nifi.web.http.port:
        nifi.web.https.host: ""
        nifi.web.https.port: 9443
        nifi.security.keystore: ./conf/keystore.jks
        nifi.security.keystoreType: jks
        nifi.security.keystorePasswd: "{{ _nftk_run_keystorepasswd }}"
        nifi.security.keyPasswd: "{{ _nftk_run_keypasswd }}"
        nifi.security.truststore: ./conf/truststore.jks
        nifi.security.truststoreType: jks
        nifi.security.truststorePasswd: "{{ _nftk_run_truststorepasswd }}"
        nifi.security.user.authorizer: "file-provider"
        nifi.security.user.login.identity.provider: ""
        nifi.cluster.protocol.is.secure: true
        nifi.cluster.node.address: ""
        nifi.cluster.node.protocol.port: 11443
      nifi_sync_dir_local: "/Users/Chilcano/1github-repo/binaries"

The vars.yml file is:

_nftk_domain: "intix.info"
_nftk_hostname: "nf[1-3]"
_nftk_dn_suffix: "OU=INTIX"
_nftk_client_cert_dn: "CN=chilcano, {{ _nftk_dn_suffix }}"
_nftk_run_clientpasswd: demo00a
_nftk_run_keypasswd: demo00b
_nftk_run_truststorepasswd: demo00c
_nftk_run_keystorepasswd: demo00d

The inventory file is:

[nifis]
nf1
nf2
nf3

nf1 ansible_host=192.168.77.5
nf2 ansible_host=192.168.77.6
nf3 ansible_host=192.168.77.7

[nifis:vars]
ansible_user=vagrant
ansible_ssh_private_key_file="/Users/Chilcano/.vagrant.d/insecure_private_key"

Updated

  • Now the Role copy binary/installer from local filesystem, if it doesn't exist, then try to download from URL. Finally downloaded binary will be copied to local filesystem to be reused.
  • The inventory file must be update by adding ansible_ssh_private_key_file in order to use synchronize Ansible module (to sync bigger files) instead of fetch Ansible module.
  • The variable nifi_sync_dir_local was added to this Ansible Role. It is the local directory where the Ansible Role will take the binary/installer to be copied to remote.

License

MIT / BSD

Author Information

This role was created in 2017 by Roger Carhuatocto, author of HolisticSecurity.io Blog.

None