The purpose of this guide is to help an IT administrator install and configure the STARRS network asset tracking application for a basic site.
Audience
The intended audience of this guide is IT administrators with Linux experience (specifically Red Hat) and a willingness to use open-source software.
Commitment
The install process should take around an hour provided appropriate resources.
Description
Definition
STARRS is a self-service network resource tracking and registration web application used to monitor resource utilization (such as IP addresses, DNS records, computers, etc). For more details, view the project description here.
Physical
STARRS requires a database host and web server to operate. It is recommended to use a single virtual appliance for all STARRS functionality.
Process
The virtual appliance is provisioned (out of scope of this guide)
Core software is installed
Dependent software packages are downloaded and installed
STARRS is acquired, configured, and installed
The web server is configured to allow access
Installation
Virtual Appliance
STARRS was tested only on RHEL-based Linux distributions. Anything RHEL6.0+ is compatible. There are no major requirements for the virtual appliance and a minimal software install will suffice. In this example we will be using a Scientific Linux 6.4 virtual machine.
Connectivity
Ensure that you are able to log into your remote system as any administrative user (in this example, root) and have internet access.
12
[root@starrs-test ~]# cat /etc/redhat-release
Scientific Linux release 6.4 (Carbon)
System Security
Firewall
Firewalls can get in the way of allowing web access to the server. Only perform these steps if you have a system firewall installed and intend on using it. In this example we will use the RHEL default iptables.
Add a rule to the system firewall to allow HTTP and HTTPS traffic to the server.
Save the firewall configuration (no restart required)
1
service iptables save
NOTE: If you have IPv6 enabled on your system, make sure to apply firewall rules to the IPv6 firewall as well.
SELinux
Any RHEL administrator has dealt with SELinux at some point in their career. There are system-wide settings that allow/deny actions by programs running on the server. Disabling SELinux is not a solution.
STARRS heavily depends on the PostgreSQL database engine for operation. PgSQL must be at version 9.0 or higher, which is NOT available from the standard RHELish repositories. PgSQL will also require the PL/Perl and PL/Python support packages (also NOT located in the repos). You will need to add new software repositories to your appliance.
Utilities
These programs will be needed at some point or another in the installation process.
Install the following packages through yum.
1
yum install cpan wget git make perl-YAML -y
We will also need the development tools group of packages installed.
1
yum groupinstall "Development tools" -y
PostgreSQL Database Engine
On your own computer, open up yum.postgresql.org and click on the latest available PostgreSQL Release link. In this case we will be using 9.2.
Locate the approprate repository link for your operating system (Fedora, CentOS, SL, etc) and architecture (i386, x86_64, etc). In this case we will be using Scientific Linux 6 - i386.
Download the package from the link you located onto the virtual appliance into any convenient directory.
Install the PostgreSQL repository package file that was downloaded with yum. Answer yes if asked for verification.
1
yum install pgdg-sl92-9.2-8.noarch.rpm
You need to ensure that the base PostgreSQL packages are hidden from future package searches and updated. Add an exclude line to your base OS repository file located in /etc/yum.repos.d/. This will prevent any PgSQL packages from being used out of the base packages.
1234567
# This example uses the sl.repo file. This will depend on your variant of OS (CentOS-base.repo for CentOS).
[sl]
name=Scientific Linux $releasever - $basearch
...
exclude=postgresql*
[sl-security]
Install the required PgSQL packages using the Yum package manager.
[root@starrs-test ~]# service postgresql-9.2 initdb
Initializing database: [ OK ]
[root@starrs-test ~]#
Make PgSQL start on system boot
1
chkconfig postgresql-9.2 on
Start the PgSQL service
123
[root@starrs-test ~]# service postgresql-9.2 start
Starting postgresql-9.2 service: [ OK ]
[root@starrs-test ~]#
su to the Postgres account and assign a password to the postgres user account using the query below. Exit back to root when done. This password should be kept secure!
123456789101112
[root@starrs-test ~]# su postgres -
bash-4.1$ psql
could not change directory to "/root"
psql (9.2.4)
Type "help" for help.
postgres=# ALTER USER postgres WITH PASSWORD 'supersecurepasswordhere';
ALTER ROLE
postgres=# \q
bash-4.1$ exit
exit
[root@starrs-test ~]#
The STARRS installer requires passwordless access to the postgres account. This is achievable by creating a .pgpass file in the root home directory.
You can replace the passwords with whatever you want. Note the passwords for later.
This file should be readable only by the user that created it. Change permissions accordingly.
1
chmod 600 .pgpass
You now need to allow users to login to the database server from the server itself. Open the /var/lib/pgsql/9.2/data/pg_hba.conf and change all methods to md5 like so:
12345678
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
This will enable login from Localhost only. If you need remote access, only allow the specific IP addresses or subnets that you need. Security is good.
Reload the postgres service to bring in the changes
1
service postgresql-9.2 reload
Create admin and client accounts for STARRS.
12345678910
[root@starrs-test ~]# psql -h localhost -U postgres
psql (9.2.4)
Type "help" for help.
postgres=# create user starrs_admin with password 'adminpass';
CREATE ROLE
postgres=# create user starrs_client with password 'clientpass';
CREATE ROLE
postgres=# \q
[root@starrs-test ~]#
Verify that you can log into the database server without being prompted for a password.
1234567
[root@starrs-test ~]# psql -h localhost -U postgres
psql (9.2.4)
Type "help" for help.
postgres=# \q
[root@starrs-test ~]#
If you get prompted for a password, STOP! You need to have this working in order to proceed. Make sure you typed everything in the file correctly and its permissions are set.
Dependencies
STARRS has many other software dependencies in order to function. These are mostly Perl modules that extend the capabilities of the language. These modules are (CPAN and package are provided however not all packages may be available):
Net::IP (perl-Net-IP)
Net::LDAP (perl-LDAP)
Net::DNS (perl-Net-DNS)
Net::SNMP (perl-Net-SNMP)
Net::SMTP (perl-Mail-Sender)
Crypt::DES (perl-Crypt-DES)
VMware::vCloud
Data::Validate::Domain
NOTE: The first time you run CPAN you will be asked some basic setup questions. Answering the defaults are fine for most installations.
Install each of these modules. Some of them are available as packages in yum.
STARRS comes in two parts: The backend (database) and the Web interface. Each one is stored in it’s own repository on Github. You will need to download both in order to use the application. Right now we will focus on the backend.
You will need a directory to store the downloaded repos in. I recommend using /opt. Clone (download) the current versions of the repositories using Git into that directory.
12345
[root@starrs-test ~]# cd /opt/
[root@starrs-test opt]# git clone https://github.com/cohoe/starrs -q
[root@starrs-test opt]# ls
starrs
[root@starrs-test opt]#
Open the installer file at /opt/starrs/Setup/Installer.pl. You will need to edit the values in the Settings section of the file to match your specific installation. Example:
dbsuperuser is the root postgres account (usually just ‘postgres’).
dbadminuser is the STARRS admin user you created above.
dbclientuser is the STARRS client user you created above.
sample will populate sample data into the database. Set to anything other than undef to enable sample content.
Run the install script
1
perl /opt/starrs/Setup/Installer.pl
A lot of text will flash across the screen. This is expected. If you see errors, then something is wrong and you should revisit the setup instructions.
You can verify that STARRS is functioning by running some simple queries.
1234567891011121314151617181920
[root@starrs-test starrs]# psql -h localhost -U postgres starrs
psql (9.2.4)
Type "help" for help.
starrs=# SELECT api.initialize('root');
NOTICE: table "user_privileges" does not exist, skipping
CONTEXT: SQL statement "DROP TABLE IF EXISTS "user_privileges""
PL/pgSQL function api.initialize(text) line 19 at SQL statement
initialize
------------------
Greetings admin!
(1 row)
starrs=# SELECT * FROM api.get_systems(NULL);
system_name | owner | group | comment | date_created | date_modified | type | os_name | last_modifier | platform_name | asset | datacenter | location
-------------+-------+-------+---------+--------------+---------------+------+---------+---------------+---------------+-------+------------+----------
(0 rows)
starrs=# \q
[root@starrs-test starrs]#
If you see “Greetings admin!” and you can perform the queries without error, then your backend is all set up and is ready for the web interface.
Apache2/httpd
The STARRS web interface requires a web server to be installed. Only Apache has been tested. If you have a new system then you might not have Apache (or in RHELish, httpd) installed.
If you do not have Apache/httpd installed, install it.
1
yum install httpd php php-pgsql -y
Navigate to the /etc/httpd/conf.d directory.
Remove the welcome.conf that exists there.
12
cd /etc/httpd/conf.d/
rm -rf welcome.conf
Enable httpd to start on boot
1
chkconfig httpd on
Start the httpd service. (Warning messages are fine, as long as the service starts you should be fine)
12345
[root@starrs-test ~]# service httpd start
Starting httpd: httpd: apr_sockaddr_info_get() failed for starrs-test.grantcohoe.com
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[ OK ]
[root@starrs-test ~]#
PHP Configuration
A minor change to the system PHP configuration allows the use of short tags for cleaner code. This feature must be enabled in the /etc/php.ini file.
In the php.ini file, set short_open_tag = on
Web Interface
You will be cloning the starrs-web into the Apache web root directory to simply deployment.
Change directory to /var/www/html/ and clone the repository. Note the . at the end of the clone command.
Copy the application/config/impulse.php.example to application/config/impulse.php
For this web environment, the defaults in this file are fine. In this file you can change which environment variable to get the current user from.
Apache needs to be given some special instructions to serve up the application correctly. Create a file at /etc/httpd/conf.d/starrs.conf with the following contents:
Restart Apache to apply the changes. (A reload is not sufficient enough)
1
service httpd restart
Testing
At this point you should have a fully functioning STARRS installation. Navigate to your server in a web browser and you should be prompted for login credentials. As we established in the authentication database file, the username is root and the password is admin. If you get the STARRS main page, then success! Otherwise start looking through log files to figure out what is wrong.
Detailed troubleshooting is out of the scope of this guide. System Administrator cleverness is a rare skill, but is the most useful thing when trying to figure out what happened. Shoot me an email if you really feel something is wrong.
]]><![CDATA[Backups for Photo Majors]]>2013-04-19T22:38:00-04:00http://www.grantcohoe.com/blog/2013/04/19/backups-for-photo-majorsData backups are something every single person should do. This presentation is with specific regard to non-technical users who want a little bit of information and some suggestions on what to do.
This presentation was never given explicitly, but it was used for a Public Speaking class.
]]><![CDATA[Impact of Disk Alignment in Virtualized Environments]]>2013-04-19T22:35:00-04:00http://www.grantcohoe.com/blog/2013/04/19/impact-of-disk-alignment-in-virtualized-environmentsThis presentation discusses the impact of disk alignment with specific regard to virtual environments. Performance can be greatly hurt by misaligned partitions both on the VM and host and all layers in between.
]]><![CDATA[CSH Wireless Network Deployment]]>2013-04-12T17:58:00-04:00http://www.grantcohoe.com/blog/2013/04/12/csh-wireless-network-deploymentWe wanted to deploy our own wireless network across the floor. I decided to make the project happen.
Requirements
Spectrum Use
We can only use three channels in the 5 GHz spectrum. RIT has a massive unified wireless deployment across all of campus. Policy states that third-parties (students, staff, etc) cannot operate wireless access points within range of the RIT network to prevent signal overlap and interference. This is not an issue in the 5 GHz spectrum, where there are many more non-overlaping channels. This limits potential clients to only those with dual-band wireless radios.
Authentication
No client-side software can be required. This is a requirement set by us to allow ease of access by users. Most operating systems support PEAP/MsCHAPv2 (via RADIUS) out of the box and require no extra configuration to make them work. Unfortunately this requires a bit more system infrastructure to make it work. Our authentication backend (MIT Kerberos) does not support this type of user authentication so we need to do some hax to make it.
Speed
We need to be faster than RIT. There is absolutely no reason to use CSH wireless over RITs unless we can be faster. By using Channel Bonding in two key areas, we can achieve this requirement and double the speed of RIT wireless. We also need to be able to do fast-reassociation between access points so that users can walk up and down floor and not lose connectivity.
Setup
Access Points
First we had to figure out where and how to deploy our range of APs. Since we have relatively few resources we used a combination of what we had lying around, purchased, and donated harware.
3x Cisco 1230
1x Cisco 1142
1x Cisco 1131
1x Cisco 1252
Looking at the map of the floor, I wanted to place the channel-bonded APs in the areas with the largest concentration of users (the Lounge and the User Center). The others would be dispersed across the other public rooms on floor.
[[ IMAGE HERE ]]
Wireless Domain Services
Cisco WDS is essentially a poor-mans controller. WDS allows you to do authentication once across your wireless domain and do relatively seamless handoff between access points. I had an extra 1230 laying around without antennas so I parked it in my server room and configured it to act as the WDS master. When a client attempts to authenticate to an AP, the auth data is sent to the WDS server where it is then processed and a response sent to the AP to let them in or not. If the client roams to another AP then the WDS server promises the new AP that the client is OK and skips the authentication phase.
This is the only device that will ever talk to the RADIUS server, so all of the configuration for that is only needed once.
NOTE: In this example the WDS server is at IP address 192.168.0.250 and the RADIUS server is at 192.168.0.100.
1234567891011121314151617181920212223242526272829
aaa new-model
!
!
aaa group server radius rad_local
server 192.168.0.250 auth-port 1812 acct-port 1813
!
aaa group server radius rad_eap
server 192.168.0.100 auth-port 1812 acct-port 1813
!
aaa authentication login eap_local group rad_local
aaa authentication login eap_methods group rad_eap
!
radius-server local
no authentication mac
nas 192.168.0.250 key 7 SUPERSECRETKEYHERE
user wds-authman nthash 7 AUTHMANPASS
!
radius-server host 192.168.0.250 auth-port 1812 acct-port 1813 key 7 SUPERSECRETRADIUSKEYHERE
radius-server host 192.168.0.100 auth-port 1812 acct-port 1813 key 7 SUPERSECRETRADIUSOTHERKEYHERE
!
!
wlccp ap username wds-authman password 7 AUTHMANPASS
wlccp authentication-server infrastructure eap_local
wlccp authentication-server client eap eap_methods
ssid prettyflyforawifi
wlccp authentication-server client leap eap_local
ssid prettyflyforawifi
wlccp wds mode wds-only
wlccp wds priority 255 interface BVI1
The access points need to use the AP username defined above to talk to the WDS server. RADIUS configuration here should be optional.
1234567891011121314151617
aaa new-model
!
aaa group server radius rad_eap
server 192.168.0.100 auth-port 1812 acct-port 1813
!
aaa authentication login default local
aaa authentication login eap_methods group rad_eap
dot11 ssid prettyflyforawifi
authentication open eap eap_methods
authentication network-eap eap_methods
authentication key-management wpa
guest-mode
!
radius-server host 192.168.0.100 auth-port 1812 acct-port 1813 key 7 SUPERSECRETRADIUSKEY
!
wlccp ap username wds-authman password 7 AUTHMANPASS
wlccp ap wds ip address 192.168.0.250
]]><![CDATA[Cross-Platform Authentication Services]]>2013-02-27T22:47:00-05:00http://www.grantcohoe.com/blog/2013/02/27/cross-platform-authentication-servicesYou are a nerd. You like doing nerd-y things. You run an entirely *nix-based network. But like all nerds, you have that one pesky Windows machine that you want to have available for your users. Or you just want the same SSO password from your Kerberos KDC. Regardless, you want Windows to talk to MIT Kerberos. Believe it or not, this is SUPPORTED!
Process Overview
We start with the user entering their credentials. These are entered in the standard Windows logon screen (which I dearly miss from Windows 2003). From there, the client is configured to have it’s default domain (realm in this case) to be the MIT Kerberos realm that your machine is a member of. This would be the equivalent of an Active Directory domain.
From here, your workstation acts just like any other Kerberized host. It uses it’s host principal (derived from what it thinks its hostname is) and configured password to authenticate to the KDC. Once it has verified it’s identity, it then goes to authenticate you. And if your password is correct, Windows lets you in!
The question is: who does it let you in as? A Kerberos principal is nothing more than a name. It is not an account, or any object that actually contains information for the system. You must map Kerberos principals to accounts. These accounts are created in Windows as either local users or via Active Directory (a whole other can of worms). Usually, you will want to create a one-to-one mapping between principal and account (grant@GRANTCOHOE.COM principal = Windows account “grant”).
KDC Setup
On your KDC, open up kadmin with your administrative principal and create a host principal for the new machine. It needs to have a password that you know, so use your favorite password generating source.
12345678910
miranda ~ # kadmin -p cohoe/admin
Authenticating as principal cohoe/admin with password.
Password for cohoe/admin@GRANTCOHOE.COM:
kadmin: ank host/caprica.grantcohoe.com@GRANTCOHOE.COM
WARNING: no policy specified for host/caprica.grantcohoe.com@GRANTCOHOE.COM; defaulting to no policy
Enter password for principal "host/caprica.grantcohoe.com@GRANTCOHOE.COM":
Re-enter password for principal "host/caprica.grantcohoe.com@GRANTCOHOE.COM":
Principal "host/caprica.grantcohoe.com@GRANTCOHOE.COM" created.
kadmin: quit
miranda ~ #
That’s it for the KDC. On to the Windows machine!
Windows 7 Client
Windows 7 (as well as Server 2008 I believe) include the basic utilities required to support MIT Kerberos. This was available for XP and Server 2003 as “Microsoft Support Tools”. The utility we will be using is called ksetup. It allows for configuration of foreign-realm authentication systems.
So to set your machine to authenticate to your already existing MIT Kerberos KDC, open up a command prompt and do the following:
After that, you need to set a Group Policy setting to automatically pick the right domain for you to login to. Open up the Group Policy Editor (gpedit.msc) and navigate to Local Computer Policy->Computer Configuration->Administrative Templates->System->Login->Assign a default domain for logon. Set this to your realm (GRANTCOHOE.COM in my case).
Do a nice reboot of your system, and you should be ready to go!
Active Directory
Active Directory can be configured to trust a foreign Kerberos realm. It will NOT synchronize information with anything, but if you just need a bunch of users to log into something and no data with it, the process is not too terrible. Rather than duplicate the information, you can find the guide that I used here: Microsoft trusting MIT Kerberos
]]><![CDATA[Stanford Webauth on Enterprise Linux]]>2013-02-13T12:38:00-05:00http://www.grantcohoe.com/blog/2013/02/13/stanford-webauth-on-enterprise-linuxBackground
Lets say you are in a domain, and you wish to access several different web services. How often do you find you have to enter the same username and password over and over again to get to each website? Wouldn’t it be nice if you could just enter it once and automagically have access to all the web services you need? Well guess what? You can!
It all works off of MIT Kerberos. Kerberos is a mechanism that centralizes your authentication to one service and allows for Single Sign-On. The concept of SSO is fairly simple: Enter your credentials once and that’s it. It save you from typing them over and over again and protects against certain attacks.
WebAuth was developed at Stanford University and brings “kerberization” to web services. All you need is some modules, a KDC, and a webserver.
In this guide, we will be utilizing the Kerberos realm/domain name of EXAMPLE.COM. We will also be using two servers:
gmcsrvx2.example.com: The Webauth WebKDC
gmcsrvx3.example.com: A secondary web server
webauth.example.com: A DNS CNAME to gmcsrvx2.example.com
Note that you will need a pre-existing Kerberos KDC in your network.
There are three components of the WebAuth system. WebAuth typically refers to the component that provides authorized access to certain content. The WebKDC is the process that handles communication with the Kerberos KDC and distributes tickets out to the client machines. The WebLogin pages are the login/logoff/password change pages that are presented to the user to enter their credentials. Certain binary packages provided by Stanford do not contain components of the WebAuth system. This is why we are going to build it from source.
Server Setup
Packages
Each machine you plan to install WebAuth on needs the following packages:
httpd-devel
krb5-devel
curl-devel
mod_ssl
mod_fcgid
perl-FCGI
perl-Template-Toolkit
perl-Crypt-SSLeay
cpan
Enterprise Linux does not include several required Perl modules for this to work. You are going to need to install them via CPAN. If you have never run CPAN before, do it once just to get the preliminary setup working.
CGI::Application::Plugin::TT
CGI::Application::Plugin::AutoRunmode
CGI::Application::Plugin::Forward
CGI::Application::Plugin::Redirect
You also need Remctl, an interface to Kerberos. For EL6 we are using remctl 2.11 available from the Fedora 15 repositories.
Open up ports 80 (HTTP) and 443 (HTTPS) since we will be doing web stuff.
NTP
Since you will be doing Kerberos stuff, you need a synchronized clock. If you do not know how to do this, see my guide on Setting up NTP.
Kerberos
Your machine needs to be a Kerberos client (meaning valid DNS, krb5.conf, and host prinicpal in its /etc/krb5.keytab). We will be using three keytabs for this application:
/etc/krb5.keytab
host/gmcsrvx2.example.com@EXAMPLE.COM
/etc/webauth/webauth.keytab
webauth/gmcsrvx2.example.com@EXAMPLE.COM
/etc/webkdc/webkdc.keytab
service/webkdc@EXAMPLE.COM
NOTE: If you wrote these in your home directory and cp’d them into their respective paths, check your SELinux contexts.
SSL Certificates
You need an SSL certificate matching the hostname of each server (gmcsrvx2.example.com, gmcsrvx3.example.com, webauth.example.com) avaiable for your Apache configuration.
IMPORTANT:You also need to ensure that your CA is trusted by Curl (the system). If in doubt, cat your CA cert into /etc/pki/tls/certs/ca-bundle.crt. You will get really odd errors if you do not do this.
Shared Libraries
We will be installing WebAuth into /usr/local and this need its libraries to be linked in the system.
We will need to run ldconfig to load in the WebAuth libraries later on.
Compile and Install WebAuth
The latest version at the time of this writing is webauth-4.1.1 and is avaiable from Stanford University. Grab the source tarball since the packages do not include features that you will need in a brand new installation.
123456789101112
[root@gmcsrvx2 ~]# tar xfz webauth-4.1.1.tar.gz
[root@gmcsrvx2 ~]# cd webauth-4.1.1
[root@gmcsrvx2 webauth-4.1.1]# ./configure --enable-webkdc
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
....
....
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing include/webauth/defines.h commands
[root@gmcsrvx2 webauth-4.1.1]#
Assuming everything is all good, go ahead and build it.
Now ensure that the new libraries get loaded by running ldconfig. This looks at the files in that ld.so.conf.d directory and adds them to the library path. MAKE SURE YOU DO THIS!!!
The Apache modules for WebAuth have been compiled into /usr/local/libexec/apache2/modules:
12
[root@gmcsrvx2 ~]# ls /usr/local/libexec/apache2/modules/
mod_webauth.la mod_webauthldap.la mod_webauthldap.so mod_webauth.so mod_webkdc.la mod_webkdc.so
mod_webauth Configuration
Make sure you have the proper keytab set up at /etc/webauth/webauth.keytab. This file can technically be located anywhere as long as you configure the module accordingly.
Next you need to create a local state directory for WebAuth. Usually this is /var/lib/webauth. The path can be changed as long as you set it in the following configuration file.
The Apache module needs a configuration file to be included in the Apache server configuration. On Enterprise Linux, this can be located in /etc/httpd/conf.d/mod_webauth.conf:
12345678910111213141516171819202122232425
# Load the module that you compiled.
LoadModule webauth_module /usr/local/libexec/apache2/modules/mod_webauth.so
# Some fancy WebAuth stuff
WebAuthKeyRingAutoUpdate on
WebAuthKeyringKeyLifetime 30d
# The path to some critical files. These should be secured.
WebAuthKeyring /var/lib/webauth/keyring
WebAuthServiceTokenCache /var/lib/webauth/service_token_cache
WebAuthCredCacheDir /var/lib/webauth/cred_cache
# The path to the keytab that webauth will use to authenticate with your KDC
WebAuthKeytab /etc/webauth/webauth.keytab
# The URL to point to if you need to login. MAKE SURE THIS IS CORRECT!
WebAuthLoginURL "https://webauth.example.com/login"
WebAuthWebKdcURL "https://webauth.example.com/webkdc-service/"
# The Kerberos principal to use when authenticating with the keytab above
WebAuthWebKdcPrincipal service/webkdc
# SSL is a good thing. Plaintext passwords are bad. Secure this server.
WebAuthSSLRedirect On
WebAuthWebKdcSSLCertFile /etc/pki/example.com/example-ca.crt
Again, adjust paths to suit your tastes. The WebAuthWebKdcSSLCertFile should be your public CA certificate that you probably cat’d earlier.
mod_webkdc Configuration
Make sure you have the proper keytab set up at /etc/webkdc/webkdc.keytab. This file can technically be located anywhere as long as you configure the module accordingly.
Next you need to create a local state directory for the WebKDC. Usually this is /var/lib/webkdc. The path can be changed as long as you set it in the configuration files.
The WebKDC utilizes an ACL to allow only certain hosts to get tickets. This goes in /etc/webkdc/token.acl:
123
# These lines allow all principals that under the webauth service to generate a token
krb5:webauth/*@EXAMPLE.COM id
krb5:webauth/gmcsrvx2.example.com@EXAMPLE.COM cred krb5 krbtgt/EXAMPLE.COM@EXAMPLE.COM
For the WebLogin pages, another configuration file is used to specify the information for it (in /etc/webkdc/webkdc.conf). This is seperate from the Apache module configuration loaded by the webserver.
12345
# The KEYRING_PATH should match what you put in your httpd config$KEYRING_PATH="/var/lib/webkdc/keyring";$URL="https://webauth.example.com/webkdc-service/";# You can make custom skins for the weblogin page. Change the path here$TEMPLATE_PATH="./generic/templates";
NOTE: You CANNOT change the location of this file or the WebKDC module will freak out.
Note that certain directives must match the Apache configuration. We will make that now in /etc/httpd/conf.d/mod_webkdc.conf:
123456789101112131415
# Load the module that you compiled.
LoadModule webkdc_module /usr/local/libexec/apache2/modules/mod_webkdc.so
# Some fancy WebKdc stuff
WebKdcServiceTokenLifetime 30d
# The path to some critical files. These should be secured.
WebKdcKeyring /var/lib/webkdc/keyring
# The path to the keytab and access control list that the webkdc will use to authenticate with your KDC
WebKdcKeytab /etc/webkdc/webkdc.keytab
WebKdcTokenAcl /etc/webkdc/token.acl
# Debugging information is wonderful. Turn this off when you get everything working.
WebKdcDebug On
Ensure that your paths match what you set up earlier.
mod_webauthldap Configuration
This step should only be done if you have an LDAP server operating in your network and can accept SASL binds. You can make certain LDAP attributes available in the web environment as server variables for your web applications. This is configured in /etc/httpd/conf.d/mod_webauthldap.conf:
12345678910111213141516171819
# Load the module that you compiled
LoadModule webauthldap_module /usr/local/libexec/apache2/modules/mod_webauthldap.so
# Webauth Keytab & credential cache file
WebAuthLdapKeytab /etc/webauth/webauth.keytab
WebAuthLdapTktCache /var/lib/webauth/krb5cc_ldap
# LDAP Host Information
WebAuthLdapHost ldap.example.com
WebAuthLdapBase ou=users,dc=example,dc=com
WebAuthLdapAuthorizationAttribute privilegeAttribute
WebAuthLdapDebug on
<Location />
WebAuthLdapAttribute givenName
WebAuthLdapAttribute sn
WebAuthLdapAttribute cn
WebAuthLdapAttribute mail
</Location>
Misc Permissions
Most of the files that you created do not have the appropriate permissions to be read/written by the webserver. Lets fix that.
I created a VHost for both “webauth.example.com” and “gmcsrvx2.example.com”, with the latter requiring user authentication to view. I name these after the hostname they are serving in /etc/httpd/conf.d/webauth.conf, Just throw a simple Hello World into the DocumentRoot that you configure for your testing host. Note that you must have NameVirtualHost-ing setup for both ports 80 and 443.
<VirtualHost *:80>
ServerName webauth.example.com
ServerAlias webauth
# Send them to somewhere useful if they request the root of this VHost
RedirectMatch permanent ^/$ https://gmcsrvx2.example.com/
# Send non-HTTPS traffic to HTTPS since we are dealing with passwords
RedirectMatch permanent ^/(.+)$ https://webauth.example.com/$1
</VirtualHost>
<VirtualHost *:443>
# Name to respond to
ServerName webauth.example.com
ServerAlias webauth
# Root directory
DocumentRoot /usr/local/share/weblogin
# SSL
SSLEngine On
SSLCertificateFile /etc/pki/example.com/webauth/host-cert.pem
SSLCertificateKeyFile /etc/pki/example.com/webauth/host-key.pem
SSLCACertificateFile /etc/pki/example.com/example-ca.crt
# Web Login directory needs some special love
<Directory "/usr/local/share/weblogin">
AllowOverride none
Options ExecCGI
AddHandler fcgid-script .fcgi
Order allow,deny
Allow from all
</Directory>
# This allows you to not need to put the file extension on the scripts
ScriptAlias /login "/usr/local/share/weblogin/login.fcgi"
ScriptAlias /logout "/usr/local/share/weblogin/logout.fcgi"
ScriptAlias /pwchange "/usr/local/share/weblogin/pwchange.fcgi"
# More special options to make things load right based on your template
Alias /images "/usr/local/share/weblogin/generic/images"
Alias /help.html "/usr/local/share/weblogin/generic/help.heml"
Alias /style.css "/usr/local/share/weblogin/generic/style.css"
# This is the actual web KDC
<Location /webkdc-service>
SetHandler webkdc
</Location>
</VirtualHost>
And the host file at /etc/httpd/conf.d/gmcsrvx2.conf:
12345678910111213141516171819202122
<VirtualHost *:80>
ServerName gmcsrvx2.example.com
ServerAlias gmcsrvx2
DocumentRoot /var/www/html
RedirectMatch permanent ^/(.+)$ https://gmcsrvx2.example.com/$1
</VirtualHost>
<VirtualHost *:443>
ServerName gmcsrvx2.example.com
ServerAlias gmcsrvx2
DocumentRoot /var/www/gmcsrvx2
SSLEngine On
SSLCertificateFile /etc/pki/example.com/gmcsrvx2/host-cert.pem
SSLCertificateKeyFile /etc/pki/example.com/gmcsrvx2/host-key.pem
SSLCACertificateFile /etc/pki/example.com/example-ca.crt
# Require a webauth valid user to access this directory
<Directory "/var/www/gmcsrvx2">
AuthType WebAuth
Require valid-user
</Directory>
</VirtualHost>
Once you are all set, start Apache. Then navigate your web browser to the host (http://gmcsrvx2.example.com). This should first redirect you to HTTPS, then bounce you to the WebLogin pages. After authenticating, you should be able to access the server.
Conclusion
This is by no means a simple thing to get set up. An intimate knowlege how how Kerberos, Apache, and LDAP work is crucial to debugging issues relating to WebAuth. All of my sample configuration files can be found in the Archive.
]]><![CDATA[IP over DNS Exploit]]>2013-01-03T01:21:00-05:00http://www.grantcohoe.com/blog/2013/01/03/ip-over-dns-exploitThe Basics
First let’s review how your traffic gets to it’s destination. You open up your favorite web browser and punch in “www.google.com”. Since your computer works with IP addresses, and not names (like people), you need to do a process of resolving the name. The Domain Name System (DNS) is the service that does this. Your ISP runs DNS servers, that are typically given to you in your DHCP lease. Your computer sends a query to this DNS server asking “what is the IP address of www.google.com”. Since your ISP does not have control over the “google.com” domain, the request is forwarded to another server. Eventually someone says “Hey! www.google.com is at 72.14.204.104”. This response is sent back through the servers to your computer.
After resolving the name, your computer now creates a data packet that will be sent to the computer at 72.14.204.104. The packet gets there by looking at your hosts route table and will hit your default gateway.
The Technology
IP over DNS is a method of encapsulating IP packets inside a DNS query. This essentially creates a VPN between a server and a client. In my case, I setup and configured the Iodine DNS server on my server located at RIT that I would use to route my traffic to the internet. Inside the tunnel exists the subnet of 192.168.0.0/27, with the server being at 192.168.0.1. My client connected and received IP address 192.168.0.2.
The Problem
Certain WLAN installations will include a guest SSID that non-controlled clients can associate to (In this scenario, I will use ExampleGuest, keeping the actual one I used anonymous). Often your traffic will get routed to a captive portal first, requiring you to enter credentials supplied by your organization. Until you do so, none of your traffic will reach it’s destination outside the LAN… UNLESS you discover a vulnerability, such as the one we are going to explore.
When I walked into the Example Corp site, I flipped open my trusty laptop and got to work. After associating with the ExampleGuest SSID, I was given the IP address 10.24.50.22/24 with gateway 10.24.50.1. I opened my web browser in attempt to get to “www.google.com”, and was immediately directed to the captive portal asking me to log in. Obviously I do not have credentials to this network, so I am at a dead end at this time.
Next I whipped out a terminal session, and did a lookup on “www.google.com”. Lo and behold the name was resolved to it’s external IP address. This means that DNS traffic was being allowed past the captive portal and out onto the network. If “www.google.com” resolved to something like “1.1.1.1” or an address in the 10.24.0.0/16 space, I know that the captive portal is grabbing all of my traffic. This would be the end of this experiment. However as I saw, this was not the case. External DNS queries were still being answered. Time to do some hacks.
The Exploit
Step 1) I need my Iodine client to be able to “query” my remote name server. I added a static route to my laptop that forced traffic to my server through the gateway given to me by the DHCP server. (ip route add 129.21.50.104 via 10.24.50.1)
Step 2) I need to establish my IPoDNS tunnel. (iodine -P mypasswordwenthere tunnel.grantcohoe.com) A successful connection gave me the IP address 192.168.0.2. I was then able to ping 192.168.0.1, which is the inside IP address of the tunnel).
Step 3) I need to change my default gateway to the server address inside the tunnel (ip route add default via 192.168.0.1).
And with that, all of my traffic is going to be encapsulated over the IP over DNS tunnel and sent to my server as DNS queries, this giving me unrestricted internet access bypassing the captive portal.
The Defense
This entire experiment would grind to a halt if DNS queries were not being handled externally. The network administrator should enable features necessary to either drop DNS requests or answer them with the captive portal IP address.
The Conclusion
End result: Unrestricted internet access over a wireless network that I have no credentials to.
Difficulty of setting this up: HIGH
Speed of tunneled internet: SLOW
Worth it for practical use: NOT AT ALL
Worth it for education: A LOT
This sort of trick can work on airplane and hotel wireless systems as well. Most sites do not think to have their captive portals capture DNS traffic in addition to regular IP traffic. As we can see here, it can be used against them.
]]><![CDATA[Dynamic Dual-Home Linux Server]]>2013-01-03T01:15:00-05:00http://www.grantcohoe.com/blog/2013/01/03/dynamic-dual-home-linux-serverRequirements
I have a shiny new VM server sitting in my dorm room. I have access to two networks, one operated by my dorm organization and the other provided to me by RIT. Both get me to the internet, but do so through different paths/SLAs. I want my server to be accessible from both networks. I also want to be able to attach VM hosts to NAT’d networks behind each respective network. This gives me a total of four possible VM networks (primary-external, primary-internal, secondary-external, secondary-internal). I have a Hurricane Electric IPv6 tunnel endpoint configured on the server, and want IPv6 connectivity available on ALL networks regardless of being external or internal. And to complicate matters, my external IP addresses are given to me via DHCP so I cannot set anything statically. Make it so.
Dependencies
IPTables
EBTables
Kernel 2.6 or newer
IPCalc
You also need access to two networks.
Script Setup
First, like any good shell script, we should define our command paths:
Now we need to load in the relevant subnet layouts. In an ideal world this would be set statically. However due to the nature of the environment I am in, both of my networks serve me my address via DHCP and is subject to change. This is very dirty and not very efficient, but it works:
12345678910111213141516171819202122
# Here we will get the subnet information for the primary network
PRIMARY_EXTERNAL_NETWORK=`$IPCALC -n $(ip -4 addr show dev $PRIMARY_EXTERNAL_INTERFACE | grep "inet" | cut -d ' ' -f 6) | cut -d = -f 2`
PRIMARY_EXTERNAL_PREFIX=`$IPCALC -p $(ip -4 addr show dev $PRIMARY_EXTERNAL_INTERFACE | grep "inet" | cut -d ' ' -f 6) | cut -d = -f 2`
PRIMARY_EXTERNAL_NETWORK="$PRIMARY_EXTERNAL_NETWORK/$PRIMARY_EXTERNAL_PREFIX"
# Then the NAT'd network behind the primary
PRIMARY_INTERNAL_NETWORK=`ip -4 addr show dev $PRIMARY_INTERNAL_INTERFACE | grep "inet" | cut -d ' ' -f 6 | sed "s/[0-9]\+\//0\//"`
# Now the subnet information for the secondary network
SECONDARY_EXTERNAL_NETWORK=`$IPCALC -n $(ip -4 addr show dev $SECONDARY_EXTERNAL_INTERFACE | grep "inet" | cut -d ' ' -f 6) | cut -d = -f 2`
SECONDARY_EXTERNAL_PREFIX=`$IPCALC -p $(ip -4 addr show dev $SECONDARY_EXTERNAL_INTERFACE | grep "inet" | cut -d ' ' -f 6) | cut -d = -f 2`
SECONDARY_EXTERNAL_NETWORK="$SECONDARY_EXTERNAL_NETWORK/$SECONDARY_EXTERNAL_PREFIX"
# And it's NAT'd network.
SECONDARY_INTERNAL_NETWORK=`ip -4 addr show dev $SECONDARY_INTERNAL_INTERFACE | grep "inet" | cut -d ' ' -f 6 | sed "s/[0-9]\+\//0\//"`
# This is where we load in the IP addresses of the interfaces
PRIMARY_EXTERNAL_IP=`ip -4 addr show dev $PRIMARY_EXTERNAL_INTERFACE | grep inet | cut -d ' ' -f 6 | sed "s/\/[0-9]\+$//"`
PRIMARY_INTERNAL_IP=`ip -4 addr show dev $PRIMARY_INTERNAL_INTERFACE | grep inet | cut -d ' ' -f 6 | sed "s/\/[0-9]\+$//"`
SECONDARY_EXTERNAL_IP=`ip -4 addr show dev $SECONDARY_EXTERNAL_INTERFACE | grep inet | cut -d ' ' -f 6 | sed "s/\/[0-9]\+$//"`
SECONDARY_INTERNAL_IP=`ip -4 addr show dev $SECONDARY_INTERNAL_INTERFACE | grep inet | cut -d ' ' -f 6 | sed "s/\/[0-9]\+$//"`
# We get the gateways by pinging once out of the appropriate interface to the multicast address of All Routers on the network.
PRIMARY_GATEWAY_IP=`ping -I $PRIMARY_EXTERNAL_IP 224.0.0.2 -c 1 | grep "icmp_seq" | cut -d : -f 1 | awk '{print $4}'`
SECONDARY_GATEWAY_IP=`ping -I $SECONDARY_EXTERNAL_IP 224.0.0.2 -c 1 | grep "icmp_seq" | cut -d : -f 1 | awk '{print $4}'`
System Configuration
In order to get most of these features working, we need to enable IP forwarding in the kernel.
123
echo "Enabling IP forwarding..."
sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv6.conf.all.forwarding=1
Routes
The routes are the most critical part of making this whole system work. We use two cool features of the linux networking stack, Route Tables and Route Rules. Route Tables are similar to your VRF tables in Cisco-land. Basically you maintain several different routing tables on a single system in addition to the main one. In order to specify what table a packet should use, you configure Route Rules. A rule basically says “Packets that match this rule should use table A”. In this system I use rules to force a packet to use a route table based on its source address.
# Kernel IP Routes
echo "Setting system default gateways"
ip route del default
ip route add default via $PRIMARY_GATEWAY_IP
ip -6 route add ::/0 dev $IPV6_TUNNEL_INTERFACE
echo "Adding default routes for primary external network..."
ip route flush table $PRIMARY_EXTERNAL_INTERFACE-routes
ip route add $PRIMARY_EXTERNAL_NETWORK dev $PRIMARY_EXTERNAL_INTERFACE src $PRIMARY_EXTERNAL_IP table $PRIMARY_EXTERNAL_INTERFACE-routes
ip route add default via $PRIMARY_GATEWAY_IP table $PRIMARY_EXTERNAL_INTERFACE-routes
echo "Adding default routes for secondary external network..."
ip route flush table $SECONDARY_EXTERNAL_INTERFACE-routes
ip route add $SECONDARY_EXTERNAL_NETWORK dev $SECONDARY_EXTERNAL_INTERFACE src $SECONDARY_EXTERNAL_IP table $SECONDARY_EXTERNAL_INTERFACE-routes
ip route add default via $SECONDARY_GATEWAY_IP table $SECONDARY_EXTERNAL_INTERFACE-routes
echo "Creating routes for primary internal network..."
ip route flush table $PRIMARY_INTERNAL_INTERFACE-routes
ip route add $PRIMARY_INTERNAL_NETWORK dev $PRIMARY_INTERNAL_INTERFACE table $PRIMARY_INTERNAL_INTERFACE-routes
ip route add default via $PRIMARY_GATEWAY_IP table $PRIMARY_INTERNAL_INTERFACE-routes
echo "Creating routes for secondary internal network..."
ip route flush table $SECONDARY_INTERNAL_INTERFACE-routes
ip route add $SECONDARY_INTERNAL_NETWORK dev $SECONDARY_INTERNAL_INTERFACE table $SECONDARY_INTERNAL_INTERFACE-routes
ip route add default via $SECONDARY_GATEWAY_IP table $SECONDARY_INTERNAL_INTERFACE-routes
echo "Creating route rules for primary external network..."
ip rule del from $PRIMARY_EXTERNAL_IP
ip rule add from $PRIMARY_EXTERNAL_IP table $PRIMARY_EXTERNAL_INTERFACE-routes
echo "Creating route rules for secondary external network..."
ip rule del from $SECONDARY_EXTERNAL_IP
ip rule add from $SECONDARY_EXTERNAL_IP table $SECONDARY_EXTERNAL_INTERFACE-routes
echo "Creating route rules for primary internal network..."
ip rule del from $PRIMARY_INTERNAL_NETWORK
ip rule add from $PRIMARY_INTERNAL_NETWORK lookup $PRIMARY_INTERNAL_INTERFACE-routes
echo "Creating route rules for secondary internal network..."
ip rule del from $SECONDARY_INTERNAL_NETWORK
ip rule add from $SECONDARY_INTERNAL_NETWORK lookup $SECONDARY_INTERNAL_INTERFACE-routes
Firewall
My script also maintains the host firewall rules, so we’ll enable those:
echo "Reseting firewall rules..."
$SERVICE iptables restart
$SERVICE ip6tables restart
echo "Setting up IPv4 firewall rules..."
$IPTABLES -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
$IPTABLES -A INPUT -p ipv6 -j ACCEPT
$IPTABLES -A INPUT -p icmp -j ACCEPT
$IPTABLES -A INPUT -i lo -j ACCEPT
$IPTABLES -A INPUT -s mason.csh.rit.edu -p tcp --dport 5666 -j ACCEPT
$IPTABLES -A INPUT -p tcp --dport 53 -j ACCEPT
$IPTABLES -A INPUT -p udp --dport 53 -j ACCEPT
$IPTABLES -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
$IPTABLES -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
$IPTABLES -A INPUT -j REJECT --reject-with icmp-host-prohibited
echo "Setting up IPv6 firewall rules..."
$IP6TABLES -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
$IP6TABLES -A INPUT -p ipv6-icmp -j ACCEPT
$IP6TABLES -A INPUT -i lo -j ACCEPT
$IP6TABLES -A INPUT -p tcp --dport 53 -j ACCEPT
$IP6TABLES -A INPUT -p udp --dport 53 -j ACCEPT
$IP6TABLES -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
$IP6TABLES -A INPUT -j REJECT --reject-with icmp6-adm-prohibited
echo "Setting up firewall rules for primary internal network..."
$IPTABLES -A FORWARD -s $PRIMARY_INTERNAL_NETWORK -j ACCEPT
$IPTABLES -t nat -A POSTROUTING -s $PRIMARY_INTERNAL_NETWORK -j SNAT --to-source $PRIMARY_EXTERNAL_IP
echo "Setting up firewall rules for secondary internal network..."
$IPTABLES -A FORWARD -s $SECONDARY_INTERNAL_NETWORK -j ACCEPT
$IPTABLES -t nat -A POSTROUTING -s $SECONDARY_INTERNAL_NETWORK -j SNAT --to-source $SECONDARY_EXTERNAL_IP
echo "Setting up default firewall actions..."
$IPTABLES -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
$IPTABLES -A FORWARD -j REJECT --reject-with icmp-port-unreachable
Router Advertisements
One of the features of my networking setup is that all networks, no matter internal or external have IPv6 connectivity. This is achieved by sending Router Advertisements out all interfaces. This is all well and good for the VMs that I am hosting, but these advertisements travel out of the physical interfaces to other hosts on the network! This is not good in that I am allowing other users to use my IPv6 tunnel interface. This puzzled me for a long time until I discovered a handy little program called “ebtables”. Ebtables is basically iptables for layer 2. As such, I was able to filter all router advertisement broadcasts out of the physical interfaces.
123456
echo "Blocking IPv6 router advertisement to the world..."
$SERVICE radvd stop
$SERVICE ebtables restart
$EBTABLES -A OUTPUT -d 33:33:0:0:0:1 -o eth1 -j DROP
$EBTABLES -A OUTPUT -d 33:33:0:0:0:1 -o eth0 -j DROP
$SERVICE radvd start
End Result
You now have a dual-homed server with two external networks, two internal networks, and IPv6 connectivity to all. Both external networks can receive their configuration via DHCP and will dynamically adjust their routing accordingly.
]]><![CDATA[Facebook Friend Guidelines]]>2013-01-03T00:42:00-05:00http://www.grantcohoe.com/blog/2013/01/03/facebook-friend-guidelinesIf I am the requester or the requestee, it means that I have had a conversation with you (Not chat/texted. Actual person-to-person) on at least two separate occasions. (Occasionally this may be reduced to one depending on circumstances) Do not get this confused with what I consider a “dialog”. A dialog is nothing more than “Hi! How are you today? What’s up?”. A conversation entails we discussed and compared our thoughts on at least one topic at length >5min. Also If you are from my present, it means that I consider there a statistically significant chance that we will see each other in person again in the future. If you are from my past, then we talked at least twice on several (>2) occasions. If you are from my future, well that feature hasn’t been implemented yet and will probably cause me to segfault.
The logic behind these guidelines is relatively simple. There is no substitute for real life interaction. The idea of “meeting someone over Facebook” doesn’t sit well with me. Whenever I see a person friend-request someone they don’t really know or have had any degree of conversation with, it makes me question their ability to do so face-to-face. It also helps avoid the potential “stalker” effect that can come about easily with social media.
Take for example Jack and Jill. Jack is an avid fan of a school sports team. Jill is his favorite player on the sports team. Jack and Jill do not know each other, and have had nothing more than a dialog (as defined earlier) between them at various meet-n-greet events. Jill gets a friend-request from Jack several days later. Jill not wanting to potentially upset Jack accepts the friend-request. She now encounters frequent chat messages from Jack that lead a quick and dull conversation. Various other actions get Jack labeled as a “stalker” in Jills mind. This is not where Jack wanted to end up as he is now considered “the bad guy”. While his motives (assume everything here to be at face-value, no hidden agendas) are honorable, they have led to an awkward situation for both of them.
By my guidelines, Jill should never have accepted the friend-request. Similarly Jack should have never sent one. Since neither of them have had any level of in-person conversations, they have no common ground on which to establish a friendship.
I will occasionally receive friend-requests from people from High School. These I find myself subjecting to a higher degree of scrutiny since I do not associate with this crowd a whole lot anymore. If I did not particularly enjoy your presence back then, I probably have no reason to change that.
My organization stores user information in an OpenLDAP directory server. This includes contact information, shell preferences, iButton IDs, etc. The directory server does not store the users password and relies on a Kerberos KDC to provide authentication via SASL. This is a widely supported configuration and is popular because you can achieve SSO (Single Sign-On) and keep passwords out of your directory. Our shell access machines are configured to do LDAP authentication of users which basically instructs PAM to do a bind to the LDAP server using the credentials provided by the user. If the bind is successful, the user logs in. If not, well sucks to be them. Before this project we had a single server handling the directory running CentOS 5.8/OpenLDAP 2.3. If this server went poof, then all userdata is lost and no one could log into any other servers (or websites if configured with Stanford Webauth. See the guide for more on this). Obviously this is a problem that needed correcting.
Technology
LDAP Synchronization
OpenLDAP 2.4 introduced a method of replication called “mirrormode”. This allows you to use their already existing syncrepl protocol to synchronize data between multiple servers. Previously (OpenLDAP <= 2.3) only allowed you to do a master-slave style of replication where the slaves are read-only and would be unwilling to perform changes. Obviously this has some limitations in that if your master (or “provider” in OpenLDAP-ish) were to die, then your users cannot do any changes until you get it back online. Using MirrorMode, you can create an “N-way Multi-Master” topology that allows all servers to be read/write and instantly replicate their changes to the others.
High Availability
To enable users to have LDAP access even if one of the servers dies, we need a way to keep the packets flowing and automatically failover to the secondary servers. Linux has a package called “Heartbeat” that allows this sort of functionality. If you are familiar with Cisco HSRP or the open-source VRRP, it works the same way. Server A is assigned an IP address (lets say 10.0.0.1) and Server B is assigned a different address (lets say 10.0.0.2). Heartbeat is configured to provide a third IP address (10.0.0.3) that will always be available between the two. On the primary server, an ethernet alias is created with the virtualized IP address. Periodic heartbeats (keep-alive packets) are sent out to the other servers to indicate “I’m still here!”. Should these messages start disappearing (like when the server dies), the secondary will notice the lack of updates and automatically create a similar alias interface on itself and assign that virtualized IP. This allows you to give your clients one host, but have it seemlessly float between several real ones.
SASL Authentication
The easy way of configuring MirrorMode requires you to store your replication DN’s credentials in plaintext in the config file. Obviously this is not very secure since you are storing passwords in plaintext. As such we can use the hosts Kerberos principal to bind to the LDAP server as the replication DN and perform all of the tasks we need to do. This is much better than plaintext!
SSL
Since we like being secure, we should really be using LDAPS (LDAP + SSL) to get our data. We will be setting this up too using our PKI. We will save this for the end since we want to ensure that our core functionality actually works first.
New Servers
I spun up two new machines, lets call them “warlock.example.com” and “ldap2.example.com”. Each of them runs my favorite Scientific Linux 6.2 but with OpenLDAP 2.4. You cannot do MirrorMode between 2.3 and 2.4.
Configuration
Packages
First we need to install the required packages for all of this nonsense to work.
openldap (LDAP libraries)
openldap-servers (Server)
openldap-clients (Client utilities)
cyrus-sasl (SASL daemon)
cyrus-sasl-ldap (LDAP authentication for SASL)
cyrus-sasl-gssapi (Kerberos authentication for SASL)
krb5-workstation (Kerberos client utilities)
heartbeat (Failover)
If you are coming off of a fresh minimal install, you might want to install openssh-clients and vim as well. Some packages may not be included in your base distribution and may need other repositories (EPEL, etc)
Kerberos Keytabs
Each host needs its own set of host/ and ldap/ principals as well as a shared one for the virtualized address. In the end, you need a keytab with the following principals:
host/ldap1.example.com@EXAMPLE.COM
ldap/ldap1.example.com@EXAMPLE.COM
host/ldap.example.com@EXAMPLE.COM
ldap/ldap.example.com@EXAMPLE.COM
WARNING: Each time you write a -randkey’d principal to a keytab, it’s KVNO (Key Version Number) is increased, thus invalidating all previous written principals. You need to merge the shared principals into each hosts keytab. See my guide on Kerberos Utilities for information on doing this.
Put this file at /etc/krb5.keytab and set an ACL on it such that the LDAP user can read it.
If you see something like “kerberos error 13” or “get_sec_context: 13” it means that someone cannot read the keytab, usually the LDAP server. Fix it.
NTP
Kerberos and the OpenLDAP synchronization require synchronized clocks. If you aren’t already set up for this, follow my guide for setting up NTP on a system before continuing.
SASL
You need saslauthd to be running on both LDAP servers. If you do not already have this set up, follow my guide for SASL Authentication before continuing.
Logging
We will be using Syslog and Logrotate to manage the logfiles for the LDAP daemon (slapd). By default it will spit out to local4. This is configurable depending on your system, but for me I am leaving it as the default. Add an entry to your rsyslog config file (usually /etc/rsyslog.conf) for slapd
1
local4.* /var/log/slapd.log
Now unless we tell logrotate to rotate this file, it will get infinitely large and cause you lots of headaches. I created a config file to automatically rotate this log according the the system defaults. This was done at /etc/logrotate.d/slapd:
123
/var/log/slapd.log {
missingok
}
Then restart rsyslog. Logrotate runs on a cron job and is not required to be restarted.
Data Directory
Since I am grabbing the data from my old LDAP server, I will not be setting up a new data directory. On the old server and on the new master, the data directory is /var/lib/ldap/. I simply scp’d all of the contents from the old server over to this directory. If you do this, I recommend stopping the old server for a moment to ensure that there are no changes occurring while you work. After scp-ing everything, make sure to chown everything to the LDAP user. I also recommend running slapindex to ensure that all data gets reindexed.
Client Library Configuration
Now to make it convenient to test your new configuration, edit your /etc/openldap/ldap.conf to change the URI and add the certificate settings.
1234
BASE dc=example,dc=com
URI ldaps://ldap.example.com
TLS_CACERT /etc/pki/tls/example-ca.crt
TLS_REQCERT allow
When configuring this file on the servers, TLS_REQCERT must be set to ALLOW since we are doing Syncrepl over LDAPS. Obviously since we are using a shared certificate for ldap.example.com, it will not match the hostname of the server and will fail. On all of your clients, they should certainly “demand” the certificate. But in this instance that prevents us from accomplishing Syncrepl over LDAPS.
Certificates for LDAPS
Since people want to be secure, OpenLDAP has the ability to do sessions inside of TLS tunnels. This works the same way HTTPS traffic does. To do this you need to have the ability to generate SSL certificates based on a given CA. This procedure varies from organization to organization. Regardless of your method, the hostname you chose is critical as this will be the name that is verified. In this setup, we are creating a host called “ldap.example.com” that all LDAP clients will be configured to use. As such the same SSL certificate for both hosts will be generated for “ldap.example.com” and placed on each server.
I placed the public certificate at /etc/pki/tls/certs/slapd.pem, the secret key in /etc/pki/tls/private/slapd.pem, and my CA certificate at /etc/pki/tls/example-ca.crt. After obtaining my files, I verify them to make sure that they will actually work:
If you are using a previous server configuration, just scp the entire /etc/openldap code over to the new servers. Make sure you blow away any files or directories that may have been added for you before you copy. If you are not, you might need to do a bit of setup. OpenLDAP 2.4 uses a new method of configuration called “cn=config”, which stores the configuration data in the LDAP database. However since this is not fully supported by most clients, I am still using the config file. For setting up a fresh install, see one of my other articles on this. (Still in progress at the time of this writing)
The following directives will need to be placed in your slapd.conf for the SSL certificates:
Depending on your system, you may need to configure the daemon to run on ldaps://. On Red-Hat based systems this is in /etc/sysconfig/ldap.
You need to add two things to your configuration for Syncrepl to function. First to your slapd.conf:
123456789101112131415
# Syncrepl ServerID
serverID 001
# Syncrepl configuration for mirroring instant replication between another
# server. The binddn should be the host/ principal of this server
# stored in the Kerberos keytab
syncrepl rid=001
provider=ldaps://ldap2.example.com
type=refreshAndPersist
retry="5 5 300 +"
searchbase="dc=example,dc=com"
attrs="*,+"
bindmethod=sasl
binddn="cn=ldap1,ou=hosts,dc=example,dc=com"
mirrormode TRUE
The serverID value must uniquely identify the server. Make sure you change it when inserting the configuration onto the secondary server. Likewise change the Syncrepl RID as well for the same reason.
Secondly you need need to allow the replication binddn full read access to all objects. This should go in your ACL file (which could be your slapd.conf, but shouldnt be).
123
access to *
by dn.exact="cn=ldap1,ou=hosts,dc=example,dc=com" read
by dn.exact="cn=ldap2,ou=hosts,dc=example,dc=com" read
WARNING: If you do not have an existing ACL setup, doing just these entries will prevent anything else from doing anything with your server. These directives are meant to be ADDED to an existing ACL.
You should be all set and ready to start the server.
123
[root@ldap1 ~]# service slapd start
Starting slapd: [ OK ]
[root@ldap1 ~]#
Make sure that when you do your SASL bind, the server reports your DN correctly. To verify this:
That DN is the one that should be in your ACL and have read access to everything.
Since the LDAP user will always need this principal, I recommend adding a cronjob to keep the ticket alive. I wrote a script that will renew your ticket and fix an SELinux permissioning problem as well. You can get it here. Just throw it into your /usr/local/sbin/ directory. There are better ways of doing this, but this one works just as well.
1
0 */2 * * * /usr/local/sbin/slaprenew
Firewall
Dont forget to open up ports 389 and 636 for LDAP and LDAPSSL!
Searching
After configuring both the master and the secondary servers, we now need to get the data initially replicated across your secondaries. Assuming you did like me and copied the data directory from an old server, your master is the only one that has real data. Start the service on this machine. If all went according to plan, you should be able to search for an object to verify that it works.
If you get errors, increase the debug level (-d 1) and work out any issues.
Replication
After doing a kinit as the LDAP user described above (host/ldap2.example.com), start the LDAP server on the secondary. If you tail the slapd log file, you should start seeing replication events occurring really fast. This is the server loading its data from the provider as specified in slapd.conf. Once it is done, try searching the server in a similar fashion as above.
Heartbeat provides failover for configured resources between several Linux systems. In this case we are going to provide high-availability of an alias IP address (10.0.0.3) which we will distribute to clients as the LDAP server (ldap.example.com). Heartbeat configuration is very simple and only requires three files:
/etc/ha.d/haresources contains the resources that we will be failing over. It should be the same on both servers.
The reason the name of the host above is not “ldap.example.com” is because the entry must be a node listed in the configuration file (below).
/etc/ha.d/authkeys contains a shared secret used to secure the heartbeat messages. This is to ensure that someone doesnt just throw up a server and claim to be a part of your cluster.
12
auth 1
1 md5 mysupersecretpassword
Make sure to chmod it to something not world-readable (600 is recommend).
Finally, /etc/ha.d/ha.cf is the Heartbeat configuration file. It should contain entries for each server in your cluster, timers, and log files.
After all this is set, start the heartbeat service. After a bit you should see another ethernet interface show up on your primary. To test failover, unplug one of the machines and see what happens!
DNS Hack
A side affect of having the LDAP server respond on the shared IP address is that it gets confused about its hostname. As such you need to edit your /etc/hosts file to point the ldap.example.com name to each of the actual host IP addresses. In this example, ldap1 is 10.0.0.1 and ldap2 is 10.0.0.2. You should modify your hosts file to include:
Likewise if you log into a client machine (in my case, I made this the KDC) you should be able to search at only the ldap.example.com host. The others will return errors.
The reason is that the client has a different view of what the server’s hostname is. This difference causes SASL to freak out and not allow you to bind.
slapd[9888]: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Wrong principal in request)
slapd[9888]: text=SASL(-13): authentication failure: GSSAPI Failure: gss_accept_sec_context
This means your DNS is not hacked or functional. Fix it according to the instructions
GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache permissions incorrect)
You need to change the SELinux context of your KRB5CC file (default is /tmp/krb5cc_$UIDOFLDAPUSER on Scientific Linux) to something the slapd process can read. Since every time you re-initialize the ticket you risk defaulting the permissions, I recommend using my script in your cronjob from above. If someone finds a better way to do this, please let me know!
If after enabling SSL on your client you receive (using -d 1):
1
TLS: can't connect: TLS error -5938:Encountered end of file.
Check your certificate files on the SERVER. Odds are you have an incorrect path.
Summary
You now have two independent LDAP servers running OpenLDAP 2.4 and synchronizing data between them. They are intended to listen on a Heartbeat-ed service IP address that will always be available even if one of the servers dies. You can also do SASL binds to each server to avoid having passwords stored in your configuration files.
If after reading this you feel there is something that can be improved or isnt clear, feel free to contact me! Also you can grab sample configuration files that I used at http://archive.grantcohoe.com/projects/ldap.
]]><![CDATA[Kerberos Utilities]]>2013-01-03T00:36:00-05:00http://www.grantcohoe.com/blog/2013/01/03/kerberos-utilitiesThere are a number of useful Kerberos client utilities that can help you when working with authentication services.
kinit
kinit will initiate a new ticket from the Kerberos system. This is how you renew your tickets to access kerberized services or renew service principals for daemons. You can kinit interactively by simply running kinit and giving it the principal you want to init as:
123
gmcsrvx1 ~ # kinit grant
Password for grant@GRANTCOHOE.COM:
gmcsrvx1 ~ #
No response is good, and you can view your initialized ticket with klist (discussed later on).
You can also kinit with a keytab by giving it the path to a keytab and a principal.
Note that it did not prompt me for a password. That is because it is using the stored principal key in the keytab to authenticate to the Kerberos server.
klist
klist is commonly used for two purposes: 1) List your current Kerberos tickets and 2) List the principals stored in a keytab. Running klist without any arguments will perform the first action.
1234567
gmcsrvx1 ~ # klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: grant@GRANTCOHOE.COM
Valid starting Expires Service principal
04/18/12 12:44:31 04/19/12 12:44:30 krbtgt/GRANTCOHOE.COM@GRANTCOHOE.COM
renew until 04/18/12 12:44:31
The duplication of the principal names represents each of the encryption types that are stored in the keytab. In my case, I use three encryption types to store my principals. If you support older deprecated enc-types (as Kerberos calls them), you will see more entries here.
kdestroy
kdestroy destroys all Kerberos tickets for your current user. You can verify this by doing:
1234
gmcsrvx1 ~ # kdestroy
gmcsrvx1 ~ # klist
klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_0)
gmcsrvx1 ~ #
ktutil
The Keytab Utility lets you view and modify keytab files. To start, you need to read in a keytab
If you want to merge two keytabs, you can repeat the read command and the second keytab will be appended to the list. You can also selectively add and delete entries to the keytab as well. Once you are done, you can write the keytab out to a file.
]]><![CDATA[High-Availability with Multipath Routing]]>2013-01-03T00:34:00-05:00http://www.grantcohoe.com/blog/2013/01/03/high-availability-with-multipath-routingMultipath Routing
Multipath routing is when you have a router with multiple equal-cost paths to a single destination. Since each of these routes carries the same weight, the router will distribute traffic across all of them roughly equally. You can read more on this technology on Wikipedia. This is more about the implementation of such a topology.
The Topology
In my setup, I have three clients on the 192.168.0.0/24 subnet with gateway 192.168.0.254. Likewise I have three servers on the 10.0.0.0/24 subnet with gateway 10.0.0.254. I want my clients to always be able to access a webserver at host 172.16.0.1/32. This is configured as a loopback address on each of the three servers. This is the multipath part of the project. Each of the servers has the same IP address on a loopback interface and will therefore answer on it. However since the shared router between these two networks has no idea the 172.16.0.1/32 network is available via the servers, my clients cannot access it. This is where the routing comes into play. I will be using an OSPF daemon on my servers to send LSA’s (read up on OSPF if you are not familiar) to the router.
Server Configuration
Firewall
You need to allow OSPF traffic to enter your system. IPtables supports the OSPF transport so you only need a minimum of one rule:
1
iptables -I INPUT 1 -p ospf -j ACCEPT
Save you firewall configuration after changing it.
Package Installation
You need to install Quagga, a suite of routing daemons that will be used to talk OSPF with the router. These packages are available in most operating systems.
Interface Configuration
Configure your main interface statically as you normally would. You also need to setup a loopback interface for the highly-available host. Procedure for this varies by operating system. Since I am using Scientific Linux, I have a configuration file in /etc/sysconfig/network-scripts/ifcfg-lo:1:
Make sure you restart your network process (or your host) after you do this. I first thought the reason this entire thing wasn’t working was a Linux-ism as my FreeBSD testing worked fine. I later discovered that it was because hot-adding an interface sometimes doesn’t add all the data we need to the kernel route table.
OSPF Configuration
Quagga’s ospfd has a single configuration file that is very Cisco-like in syntax. It is located at /etc/quagga/ospfd.conf:
1234567
hostname server1
interface eth0
router ospf
ospf router-id 10.0.0.1
network 10.0.0.0/24 area 0
network 172.16.0.1/32 area 0
log stdout
ospfd has a partner program called zebra that manages the kernel route table entries learned via OSPF. While we may not necessarily want these, zebra must be running and configured in order for this to work. Edit your /etc/quagga/zebra.conf:
12
hostname server1
interface eth0
After you are all set, start zebra and ospfd (and make sure they are set to come up on boot).
Router Configuration
I am using a virtualized Cisco 7200 router in my testing environment. You may have to modify your interfaces, but otherwise this configuration is generic across all IOS:
123456
R1>enable
R1#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)#router ospf 1
R1(config-router)#network 10.0.0.0 0.0.0.255 area 0
R1(config-router)#end
This enables OSPF on the 10.0.0.0/24 network and will start listening for LSAs. When it processes a new one, the route will be inserted into the routing table. After a while, you should see something like this:
12345
R1#show ip route ospf
172.16.0.0/32 is subnetted, 1 subnets
O 172.16.0.1 [110/11] via 10.0.0.3, 00:04:42, GigabitEthernet1/0
[110/11] via 10.0.0.2, 00:04:42, GigabitEthernet1/0
[110/11] via 10.0.0.1, 00:04:42, GigabitEthernet1/0
This is multipath routing. The host 172.16.0.1/32 has three equal paths available to get to it.
Testing
To verify that my clients were actually getting load-balanced, I threw up an Apache webserver on each server with a static index.html file containing the ID of the server (server 1, server 2, server 3, etc). This way I can curl the highly-available IP address and can see which host I am getting routed to. You should see your clients get distributed out across any combination of the three servers.
Now lets simulate a host being taken offline for maintenance. Rather than having to fail over a bunch of services, all you need to do is stop the ospfd process on the server. After the OSPF dead timer expires, the route will be removed and no more traffic will be routed to it. That host is now available for downtime without affecting any of the others. The router just redistributes the traffic on the remaining routes.
But lets say the host immediately dies. There is no notification to the router that the backend host is no longer responding. The only thing you can do at this point is wait for the OSPF dead timer to expire. You can configure this to be a shorter amount of time, thus making “failover” more transparent.
Summary
Multipath routing is basically a poor-mans load balancer. It has no checks to see if the services are actually running and responding appropriately. It also does not behave very well with sessions or data that involves lots of writes. However for static web hosting or for caching nameservers (the use that sparked my interest in this technology), it works quite well.
]]><![CDATA[Secure OpenLDAP Server with Kerberos Authentication]]>2013-01-03T00:26:00-05:00http://www.grantcohoe.com/blog/2013/01/03/secure-openldap-server-with-kerberos-authenticationHost Configuration
There are some things we need to set up prior to installing and configuring OpenLDAP.
Kerberos
You will need a working KDC somewhere in your domain. You also need to have the server configured as a Kerberos client (/etc/krb5.conf) and be able to kinit without issue.
There are two principals that you will need in your /etc/krb5.keytab:
host/server.example.com@EXAMPLE.COM
ldap/server.example.com@EXAMPLE.COM
saslauthd
If you do not already have this working, see my guide on how to set it up.
Packages
You need the following for our setup:
krb5-server-ldap (for the Kerberos schema)
openldap
openldap-clients
openldap-servers
cyrus-sasl-gssapi
cyrus-sasl-ldap
SSL Certificates
You will need SSL certificates matching the hostname you intend your LDAP server to listen on (ldap.example.com is different than server.example.com). Procure these from your PKI administrator. I place mine in the default directories as shown:
Server Public Key - /etc/pki/tls/certs/slapd.pem
Server Private Key - /etc/pki/tls/private/slapd.pem
CA Certificate - /etc/pki/tls/cert.pem
ACLs
The LDAP user needs to be able to read the private key and keytab you configured earlier. Ensure that it can.
By default there are some extra files in the configuration directory. Since we are not using the cn=config method of configuring the server, we are simply going to remove the things we don’t want.
We will be re-creating ldap.conf later on with the settings that we want.
Kerberos Schema
Copy /usr/share/doc/krb5-server-ldap-1.9/kerberos.schema into your schema directory (default: /etc/openldap/schema. This contains all of the definitions for Kerberos-related attributes.
Data Directory
Decide where you want your data directory to be. By default this is /var/lib/ldap Copy over the standard DB_CONFIG file to that directory from /usr/share/openldap-servers/DB_CONFIG.example. This contains optimization information about the structure of the database.
Enable LDAPS
Some systems require you to explicitly enable LDAPS listening. On Red Hat Enterprise Linux-based distributions, this is done by changing SLAPD_LDAPS to YES in /etc/sysconfig/ldap.
slapd.conf
Create a /etc/openldap/slapd.conf to configure your server. You can grab a copy of my basic configuration file here.
# Include base schema files provided by the openldap-servers package.
include /etc/openldap/schema/corba.schema
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/duaconf.schema
include /etc/openldap/schema/dyngroup.schema
include /etc/openldap/schema/inetorgperson.schema
include /etc/openldap/schema/java.schema
include /etc/openldap/schema/misc.schema
include /etc/openldap/schema/nis.schema
include /etc/openldap/schema/openldap.schema
include /etc/openldap/schema/pmi.schema
include /etc/openldap/schema/ppolicy.schema
# Site-specific schema and ACL includes. These are either third-party or custom.
include /etc/openldap/schema/kerberos.schema
# Daemon files needed for the running of the daemon
pidfile /var/run/openldap/slapd.pid
argsfile /var/run/openldap/slapd.args
# Limit SASL options to only GSSAPI and not other client-favorites. Apparently there is an issue where
# clients will default to non-working SASL mechanisms and will make you angry.
sasl-secprops noanonymous,noplain,noactive
# SASL connection information. The realm should be your Kerberos realm as configured for the system. The
# host should be the LEGITIMATE hostname of this server
sasl-realm EXAMPLE.COM
sasl-host ldap.example.com
# SSL certificate file paths
TLSCertificateFile /etc/pki/tls/certs/slapd.pem
TLSCertificateKeyFile /etc/pki/tls/private/slapd.pem
TLSCACertificateFile /etc/pki/tls/cert.pem
# Rewrite certain SASL bind DNs to more readable ones. Otherwise you bind as some crazy default
# that ends up in a different base than your actual one. This uses regex to rewrite that weird
# DN and make it become one that you can put within your suffix.
authz-policy from
authz-regexp "^uid=[^,/]+/admin,cn=example\.com,cn=gssapi,cn=auth" "cn=ldaproot,dc=example,dc=com"
authz-regexp "^uid=host/([^,]+)\.example\.com,cn=example\.com,cn=gssapi,cn=auth" "cn=$1,ou=hosts,dc=example,dc=com"
authz-regexp "^uid=([^,]+),cn=example\.com,cn=gssapi,cn=auth" "uid=$1,ou=users,dc=example,dc=com"
# Logging
#loglevel 16384
loglevel 256
# Actual LDAP database for things you want
database bdb
suffix "dc=example,dc=com"
rootdn "cn=ldaproot,dc=example,dc=com"
rootpw {MD5}X03MO1qnZdYdgyfeuILPmQ==
directory /var/lib/ldap
# Indicies for the database. These are used to improve performance of the database
index entryCSN eq
index entryUUID eq
index objectClass eq,pres
index ou,cn,mail eq,pres,sub,approx
index uidNumber,gidNumber,loginShell eq,pres
# Configuration database
database config
rootdn "cn=ldaproot,dc=example,dc=com"
# Monitoring database
database monitor
rootdn "cn=ldaproot,dc=example,dc=com"
To generate a password for your directory and hash it, you can use the slappasswd utility. See my guide on LDAP utilities for details. After that you should be all set. Start the slapd service, but dont query it yet.
Client Configuration
The /etc/openldap/ldap.conf file is the system-wide default LDAP connection settings. This way you do not have to specify the host and protocol each time you want to run a command. Make it now.
12345
# OpenLDAP client configuration file. Used for host default settings
BASE dc=example,dc=com
URI ldaps://ldap.example.com
TLS_CACERT /etc/pki/tls/cert.pem
TLS_REQCERT demand
Population
Right now you should have a running server, but with no data in it. I created a simple example setup file to get a basic tree set up. At this point you need to architect how you want your directory to be organized. You may chose to follow this or chose your own.
]]><![CDATA[Fix Me Maybe]]>2013-01-03T00:23:00-05:00http://www.grantcohoe.com/blog/2013/01/03/fix-me-maybeFix Me Maybe (parody of Call Me Maybe)
Words by Grant Cohoe (cohoe)
Music & original words by Carly Rae Jepsen
This was dedicated to Ross Delinger (rossdylan) because: “FIX THE DAMN IRC SERVER!” Like the original song, it then went a lot further than expected. What more might come of this?!
I started as a project
Something come Fall we'd forget
The system full of neglect
But now you've logged into me.
Everything's been broke for awhile
It's not great R-T-P style
This wasn't supposed to be a trial
But now you've logged into me.
Your session was open
Bytes of logs were flowin'
Files were c-h-ownin'
Need to get them daemons going
I'm used to broken things
These users are crazy
But here's my address,
so fix me maybe?
It's hard to figure out
What is wrong here
But here's my address,
so fix me maybe?
I'm used to broken things
These users are crazy
But here's my address,
so fix me maybe?
And all the other boys
Try and break me
But here's my address,
so fix me maybe?
You took your time with fix
Broke in no time 'cause they're pricks
You didn't help anything
But still, you're logged in me
I page, and cache and write
Every morn day and night
Processor cycles aren't tight
But still, you're logged in me
Your session was open
Bytes of logs were flowin'
Files were c-h-ownin'
Need to get them daemons going
I'm used to broken things
These users are crazy
But here's my address,
so fix me maybe?
It's hard to figure out
What is wrong here
But here's my address,
so fix me maybe?
I'm used to broken things
These users are crazy
But here's my address,
so fix me maybe?
And all the other boys
Try and break me
But here's my address,
so fix me maybe?
Before you came into my logs
I was broken so bad
I was broken so bad
I was broken so so bad
Before you came into my logs
I was broken so bad
And you should know that
I was broken so bad
It's hard to figure out
What is wrong here
But here's my address,
so fix me maybe?
I'm used to broken things
These users are crazy
But here's my address,
so fix me maybe?
And all the other boys
Try and break me
But here's my address,
so fix me maybe?
Before you came into my logs
I was broken so bad
I was broken so bad
I was broken so so bad
Before you came into my logs
I was broken so bad
And you should know that
So fix me, maybe?
]]><![CDATA[Makefile-based PKI Management]]>2013-01-01T23:03:00-05:00http://www.grantcohoe.com/blog/2013/01/01/makefile-based-pki-managementUsing a couple of seed files, you can easily get started managing your own PKI for your network. You need to have OpenSSL and Make installed on the system you are working with.
First, pick a directory to store your certificates in. I am going to use /etc/pki/example.com since this is where all the other system SSL stuff is stored and for you SELinux people, will already have the right contexts. cd into that directory and create the Makefile:
This lets us perform two actions (targets): Issue and Revoke. Issue will create a new host certificate from your CA. Revoke will expire an already-existing certificate. This is mainly used for renewal since you need to revoke then issue.
Next we will need a configuration directory and some basic files:
This will generate a CA certificate based on your settings file and will last for 5 years, after which you will need to renew it.
Enter a certificate password when prompted. You do not need to specify a Common Name (CN) as asked later in the process. Ensure that the other field defaults are correct, as they came from your openssl.cnf file and are needed to match in other certificates.
1234567891011121314151617181920212223
[root@gmcsrvx1 config]# openssl req -config openssl.cnf -new -x509 -extensions v3_ca -keyout CA/example-ca.key -out CA/example-ca.crt -days 1825Generating a 2048 bit RSA private key..................................................................................................+++writing new private key to 'CA/example-ca.key'Enter PEM pass phrase:Verifying - Enter PEM pass phrase:-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Organization Name (company) [Example Corp]:Organizational Unit Name (department, division) [System Engineering]:Email [root@example.com]:Locality Name (city) [Rochester]:State or Province Name (full name) [New York]:Country Name (2 letter code) [US]:Common Name (full hostname) []:[root@gmcsrvx1 config]#
Unfortunately this process creates the private key file to be world readable. This is extremely bad. Fix it. Also ensure that the public key is readable by all. It is supposed to be.
Now back up one directory (back to the working directory you started at). I usually make a symlink to the CA public key here so you can easily reference it.
You can now issue SSL certificates by typing make DEST=hostname where hostname is the name of the server you are going to issue to.
[root@gmcsrvx1 example.com]# make DEST=gmcsrvx1mkdir -p gmcsrvx1openssl req -new -nodes -out gmcsrvx1/req.pem -config config/openssl.cnfGenerating a 2048 bit RSA private key..............................................+++......................................+++writing new private key to 'host-key.pem'-----You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Organization Name (company) [Example Corp]:Organizational Unit Name (department, division) [System Engineering]:Email [root@example.com]:Locality Name (city) [Rochester]:State or Province Name (full name) [New York]:Country Name (2 letter code) [US]:Common Name (full hostname) []:gmcsrvx1.example.commv host-key.pem gmcsrvx1/openssl ca -out gmcsrvx1/host-cert.pem -config config/openssl.cnf -infiles gmcsrUsing configuration from config/openssl.cnfEnter pass phrase for /etc/pki/example.com/config/CA/example-ca.key:Check that the request matches the signatureSignature okThe Subject's Distinguished Name is as followsorganizationName :PRINTABLE:'Example Corp'organizationalUnitName:PRINTABLE:'System Engineering'localityName :PRINTABLE:'Rochester'stateOrProvinceName :PRINTABLE:'New York'countryName :PRINTABLE:'US'commonName :PRINTABLE:'gmcsrvx1.example.com'Certificate is to be certified until May 1 18:12:03 2013 GMT (365 days)Sign the certificate? [y/n]:y1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base Updatedif [ gmcsrvx1 = mail -o gmcsrvx1 = mail/ ]; then cp gmcsrvx1/host-cert.pem gmcsr.pem; fichmod go= gmcsrvx1/host-key.pemif [ -e gmcsrvx1/host-combined.pem ]; then chmod go= gmcsrvx1/host-combined.pem;[root@gmcsrvx1 example.com]#
If you look in the directory it made for you (the hostname you specified) you will find the private and public keys for the server.
12345
[root@gmcsrvx1 example.com]# ls -l gmcsrvx1/total 12-rw-r--r--. 1 root root 3997 May 1 14:12 host-cert.pem-rw-------. 1 root root 1704 May 1 14:12 host-key.pem-rw-r--r--. 1 root root 1171 May 1 14:12 req.pem
You can revoke this certificate using the makefile as well:
1234567
[root@gmcsrvx1 example.com]# make revoke SOURCE=gmcsrvx1openssl ca -config config/openssl.cnf -revoke gmcsrvx1/host-cert.pemUsing configuration from config/openssl.cnfEnter pass phrase for /etc/pki/example.com/config/CA/example-ca.key:Revoking Certificate 01.Data Base Updated[root@gmcsrvx1 example.com]#
There you go, yet another useful service to have around for dealing with your network. Raw files used above are available in the Archive. Now back to Webauth with me…
]]><![CDATA[System Technology Accounting and Resource Registration Solution (STARRS)]]>2013-01-01T20:10:00-05:00http://www.grantcohoe.com/blog/2013/01/01/system-technology-accounting-and-resource-registration-solution-starrsBackground
Like most of my projects, it all started with CSH. RIT allocates us two /24 public-facing networks to distribute out to our users. These resources need to have some degree of accounting in the event a user does something stupid (piracy, kiddie-pr0n, etc). RIT handles this with their own internal application, referred to as “start.rit.edu”. Fun aside, Start.RIT was coded by an old CSHer, now RIT employee and CSH advisor. He still maintains portions of it today. When the CSH network got to the point of needing our own internal application, a member (Joe Sunday) created “start.csh.rit.edu”. Similar to it’s RIT counterpart, start.csh allowed administrators to register machines and distribute resources on the network. Start.csh also allowed for users to manage firewall rules for their hosts at the border firewall systems. On the backend, it had a script that would automatically generate an ISC-DHCPD config file and load it into the server.
Unfortunately over time, bits of Start began to break. The fact that users could not register their own machines was a major source of irritation for the RTPs (Root-Type-Persons, or Sysadmins). When the network topology was shifted to accomodate some reorganization within RIT, the firewall rule system broke completely. There was no way to clear out old hosts that havent been on the network in years. And after the house DHCP server was compromised (then obliterated by the RTPs for security), the automatic DHCP generation was completely gone. We needed something new.
Starting in the Spring of 2011 I starting to architect a new application to encoumpase the previous functionality and add a lot of new features and enhancements that would benefit us for years to come. Going into the projects, I knew one thing: It had to be based in PostgreSQL. Pg has native datatypes for IP addresses, subnets, and MAC addresses. I know that this would be invaluable to have later on in the project. I also knew that I didnt want to write a daemon that ran on some server imposing application logic.
Development
At this point in my career, I didn’t have a whole lot of database architecting experience. I began to search for a schema creation application to help me architect what I knew was going to be a complex schema. I ran across a service called SchemaBank. (SB is now defunct) It was a schema design webapp that supported Postgres. It also introduced me to two things that would change the course of the application forever: Triggers and Functions. I realized that I could use Pg as the backend daemon to the entire application. For client interaction with the application, I started to develop a set of wrapper API functions that would help impose the application logic on the stored data. While most of this was taken care of in the relations between entities, there were some that could not be expressed as a relation. Fast forward two months, and I had a first revision of the database and the API to start writing an interface for.
At this point I didn’t know any sort of web languages other than HTML and CSS either. So using a book I won at BarCampRoc on PHP, MySQL, and Javascript (a very good book by the way), I started to learn PHP. Originally I was going to go with a purely non-OOP model for the webapp, however after consulting one of my web-dev friends (Ben Russell), he suggested that I go with an OOP model and to use some sort of framework to avoid having to write a ton of extra code. At this point I went with one that another one of my web-dev friends had used, Codeigniter. So I set to work and started writing classes. During this time, I knew that I needed a new a new dhcpd.conf generation function. One of my friends (Anthony Gargiulo) expressed an interest in helping out with the project, so I tasked him with writing a generation function in Perl. Four months later, I had a working PHP web interface (and he had a working dhcpd.conf generator). At this point, I had everything except a name. I dont really remember a lot about how it came about, but I think I took the first cool-sounding word that came to my head. I was in a Quake 3 Arena mood that day and remembered an old console command (impulse). I took that word and backronymed it to the IP Management Program for Use in Local Server Environments.
Initial Implementation
When we returned to RIT in the Fall, I set to work deploying IMPULSE. After a few hickups here and there, everything was in place and in use. Users were fairly receptive to it, mainly that they could register their devices themselves. However after some time, a few problems emerged. The web interface became terribly slow with all of the data that people had entered. It wasnt very efficient at allowing users to complete basic tasks and had a few bugs. Clearly some changes needed to occur, but during the course of the year I didn’t want to touch it. I spent 6 months writing code on this project, and I really didnt want to work on it again.
Refresh
My latest co-op offerred me a chance to take to the code once again, this time with a new use case in mind. They wanted something that could manage network registrations for developers in a complex environment, but there were a few major additions that needed to happen. IMPULSE needed to go global. It needeed more specific access control. It needed better device integration. And most importantly, it also needed a new interface. Armed with all the knowlege I have gained over the last year, I set out to work. 2 months later I had a new web interface and a lot of new fixes and enhancements to the core. However the name IMPULSE no longer applied. So I cooked up a new one: STARRS (System Technology Accounting and Resource Registration Solution). And that’s presumably what you actually care about.
Features
STARRS starts off with Datacenters, which correspond to your physical locations. Each datacenter contains Subnets and VLANs. Each datacenter then contains one or more Availability Zones, which are logical partitions of resources within the datacenter. AZs contain ranges of your subnets.
Within datacenters are Systems. A system is a computer, based on a hardware Platform (usually Custom though). A system can have Interfaces which attach to your network. On each interface can be Addresses that are assigned from your Availability Zones. You can get your addresses in a variety of ways (depending on family). DHCP and Static for IPv4, and Autoconf, DHCPv6, and Static for IPv6. (Oh yeah, STARRS does both IPv4 and IPv6). Your addresses are given to you with a set expiration date. If you dont renew before that date (You will get email notifications), it will be automatically deleted. On your addresses, you can create a variety of DNS Records that can be added into managed DNS Zones. Zone DDNS updates are done with Keys that are stored within STARRS. If your address is configured with DHCP, you can be a member of a DHCP Class. Classes, Ranges, and Subnets all have a variety of configuration options that can be set. You can also set options globally. For network systems, you can store SNMP Credentials that STARRS can use to look up Switchport and CAM Table data. Using this you can easily locate your system interfaces in your infrastructure. And of course, you can search for systems that you have configured.
STARRS depends on your existing authentication infrastructure for its user data. It does not track credentials within itself, but relies on the client to provide the user and external sources for privileging. The STARRS web interface gets the username from the web server from configurable PHP variables. It takes the username and calls an API initializing function that sets up your privilege levels. STARRS can get privilege data from one of the following:
Local Tables
OpenLDAP
Active Directory
Users can be global ADMINs and have RW access to everything in the database. They can be USERs who can create/edit/remove their own stuff, but cannot see certain confidential configuration and cannot edit other peoples stuff. People who are Group Admins can edit other peoples stuff in their group. There is also a PROGRAM user-level that allows external applications Read-Only access to user resources. If you don’t exist in the privilege source, you will be unable to initialize and do anything in STARRS.
STARRS also has the ability to automatically generate a configuration file for the ISC-DHCPD server based on the resources you configure. It supports key-based DDNS updates to zones, DHCP classes, global/range/subnet/class options, and more. A cronjob runs to periodically generate, download, and reload the server configuration file. Eventually this might change to something more real, but for now this is what works.
Use cases have STARRS working with the following web authentication services:
There is an unofficial and unsupported command-line interface written by a friend of mine (Ryan Brown) located Here.
STARRS is licensed under the MIT license.
Demo
You can access the demo by clicking here. Username is ‘root’ and password is ‘admin’. The database resets itself every 24 hours at midnight, so don’t expect long-term persistence of data.
Who Should Use This
If you are a service provider to a group of users who consume network resources that you control. You want some way of accounting for what resources are in use and where they are in your network. The use cases of it so far are:
Computer Science House - Dorm of college students (about 350 systems)
Work - Product developers in 4 sites across 3 countries (1000+ systems)
Enterprise - My personal VM server (30 systems)
Future
There are a couple of enhancements on deck for the next release (whever that may be). See the respective Github project issues for details. However STARRS will soon be encompasing libvirt support. When you create a system, you can specify it to be a Virtual Machine living on a Host in one of your datacenters. None of this has been worked on, but the idea is for STARRS to be a complete network management solution. Maybe I’ll call it CloudSTARRS (like rockstars, but in the cloud haha).
