User Management Using HUE Shell (CDH)

In this short post I will focus on user management aspects of HUE. Something that every administrator needs to tackle.

Intro

When it comes to production setups HUE provides ways to integrate existing user base (e.g LDAP) with the service itself. That pretty much solves the problem for production. The situation looks a bit different in a testing or teaching environments for which you could require more user accounts and no backend is available. The simplest way would be to create them manually, but that takes time, becomes tedious, and I’ve been there. Clicking for half an hour just to create some users is not productive at all. There must be other way!

LDAP Integration

But let’s start with few words about LDAP integration. Cloudera provides detailed instructions on how to configure HUE in order to integrate it with existing user base. When you set up this correctly you’re almost done. The last thing you need to know about this integration is that every time you want to sync LDAP with HUE database you need to invoke the sync operation manually. You can do it from HUE web interface. Better solution is to have a script which can be run each time you onboard new users or groups for HUE. There are couple of HUE shell commands which help you achieve this goal. You need to invoke them from HUE directory.

Here are exemplary situations you may find yourself in:

If you want to add a user who is part of LDAP group which you previously added to HUE, you would run:

${HUE_PATH} import_ldap_user ${user}

Then you may want to add a new LDAP group to HUE database:

${HUE_PATH} import_ldap_group ${group}

Third situation is when you need to sync LDAP and HUE when you already have users and groups in HUE database but you have changed something in LDAP (e.g you have removed couple of users from developer group and you want HUE to be aware of that change):

${HUE_PATH} sync_ldap_users_and_groups

Without LDAP

Now, let’s take a look at our problematic scenario. HUE shell provides the ability to run commands from a script. That’s something we can work with! If you use external HUE database for your testing purposes you’re set to go. You know the database username/password and you can access HUE shell without issue. The problem occurs when you use internal database (as in most testing setups). Then you need a way to get to the HUE database password (in modern versions of HUE the internal db password is encrypted and is not easily available from a config file as it used to be.

Let’s get this password using Cloudera Manager API. Here’s a little script to help with that. You need cm_api libraries for python.

#!/usr/bin/env python


from cm_api.endpoints.role_config_groups import get_all_role_config_groups

from cm_api.api_client import ApiResource


api = ApiResource(cm_host, username=username, password=password)


CLUSTER = None

for cluster in api.get_all_clusters():

   CLUSTER = cluster



for service in CLUSTER.get_all_services():

      if service.name == ‘hue’’:

          for value in service.get_config(view='full'):

              if 'database_password' in value:

                  hue_db_pass = str(value[‘database_password’]).split(‘ ‘)[-1]

You can easily write the password to a file

hue_db_pass_file = open(‘hue-db-pass’, ‘w’)

hue_db_pass_file.write(hue_db_pass)

hue_db_pass_file.close()

Just provide ‘api’ object with values of the Cloudera Manager host address, the username (needs to be administrator, otherwise you won’t be able to get the password) as well as the password.

With HUE database password we can proceed to HUE shell to do our job. For this we will use a shell script:

#!/bin/bash

INSTALL_DIR=’opt/cloudera/parcels/CDH/lib/’
USER=$1

PASSWORD=$2

HUE_DB_PASS=$3


export HUE_CONF_DIR="/var/run/cloudera-scm-agent/process/$(ls) -alrt /var/run/cloudera-scm-agent/process | grep HUE | tail -1 | awk '{print $9}'`"

export HUE_IGNORE_PASSWORD_SCRIPT_ERRORS=1

export HUE_DATABASE_PASSWORD=$HUE_DB_PASS


${INSTALL_DIR}/hue/build/env/bin/hue shell <<EOF

from django.contrib.auth.models import User

from django.contrib.auth.models import Group

user = User.objects.create(username='${USER}')

user.set_password('${PASSWORD}')

user.is_active = True

user.is_superuser = True

group_obj = Group.objects.get(name="default")

user.groups.add(group_obj)

user.save()

EOF

With this script we can easily create user in HUE without going into web interface. Just use this script with a list of users and you saved yourself some dull clicking.

HUE User Groups and Permissions Overview

We talked a bit about adding users or groups to HUE, but there is one more important thing we should take care of – permissions. The most obvious way to manage permissions is to use web UI. Too obvious 🙂 Let’s take a look on how we can utilize shell to do this.

from useradmin.models import HuePermission, GroupPermission

from django.contrib.auth.models import Group


permission = HuePermission.objects.all()

for perm in permission:

   print perm

This code will list all the available permissions that we can assign to a group. Something like:

about.access:Launch this application(1)

beeswax.access:Launch this application(2)

filebrowser.access:Launch this application(3)

help.access:Launch this application(4)

jobbrowser.access:Launch this application(5)

jobsub.access:Launch this application(6)

metastore.write:Allow DDL operations. Need the app access too.(7)

metastore.access:Launch this application(8)

oozie.dashboard_jobs_access:Oozie Dashboard read-only user for all jobs(9)

oozie.access:Launch this application(10)

oozie.disable_editor_access:Disable Oozie Editor access(11)

As you can see the permission consists of three parts. Application name (e.g ‘about), action (e.g ‘access’) and description (e.g Launch this application). Using this information we can add permission to a group. To make things more interesting we will create a new group. Then we will add ‘access’ action to that group.

newgroup = Group.objects.create(name=’Test’)

permission = HuePermission.objects.create(app=’about’,action=’access’)


GroupPermission.objects.create(group=newgroup,hue_permission=permission)

And that’s it! With this simple steps you can create a group and edit its permissions. The method to add user to groups is mentioned in the user creation script. Mainly the part:

group_obj = Group.objects.get(name="default")

user.groups.add(group_obj)

Summary

As you can see HUE shell provides a nice way to help you manage the users when there is no possibility to use anything like LDAP. The big disadvantage is that there is little to none documentation and pretty much all things you read here had been gathered from scarce information available online (other people experiences) or discovered by trial and error method. Hopefully this post will save you some precious time.

Disclaimer
The code presented in this post was tested with HUE 3.10 and cm_api API version 13 (CM 5.8.0)

Post by Piotr Bednarek

Piotr is a system administrator with passion for Open Source. He has gained his first experience with distributed systems working for companies like GarageFarm.NET and CopernicusComputing, where he helped to build and maintain HA infrastructure. Now he works through Hadoop ecosystem and supports colleagues from GetInData.

Leave a Reply

Your email address will not be published. Required fields are marked *

Blue Captcha Image
Refresh

*