Squid Proxy with NTLM Authentication, SquidGuard and BerkeleyDB
Babu Ram Dawadi, System Administrator & SOI-Asia Operator,
Center for Information Technology (CIT), IOE Pulchowk, TU Nepal
Last Updated: June 2007
Background
A server that acts as an intermediary between a workstation user and the Internet so that the enterprise can ensure security, administrative control, and caching service. It’s a technique used to cache information on a Web server and acts as an intermediary between a Web client and that Web server.
A proxy server may service requests without contacting the specified server, by retrieving content saved from a previous request, made by the same client or even other clients. This is called caching. Caching proxies keep local copies of frequently requested resources, allowing large organizations and Internet Service Providers to significantly reduce their upstream bandwidth usage and cost, while significantly increasing performance
Web proxy
Proxies that focus on WWW traffic are called web proxies. Many web proxies attempt to block offensive web content. Other web proxies reformat web pages for a specific purpose or audience. Network/System Administrators can also deploy proxies to intercept computer viruses and other hostile content served from remote web pages
Many organizations including schools, campuses, corporations, and countries use proxy servers to enforce acceptable network use policies or to provide security, anti-malware and/or caching services. A traditional web proxy is not transparent to the client application, which must be configured to use the proxy (manually or with a configuration script). In some cases, where alternative means of connection to the Internet are available (e.g., a SOCKS server or NAT connection), the user may be able to avoid policy control by simply resetting the client configuration and bypassing the proxy. Furthermore administration of browser configuration can be a burden for network/system administrators
Access control: Some proxy servers implement a logon requirement. In large organizations, authorized users must log on to gain access to the WWW. The organization can thus track usage to individuals.
Introduction
This document details the necessary steps to install and configure an open source proxy server including SAMBA, WINBIND, SQUIDGUARD & BERKELEYDB that allows the NTLM authentication against NT Domain controllers. It helps to increase user navigation security using activities logging, users and gropus access control lists, blocking unnecessary sites and bandwith tuning.
To achieve this, I have selected the packages Squid, Samba, Winbind, SquidGuard and Berkeley DB with WinNT domain controller.
NTLM Authentication
NTLM is an authentication protocol used in various Microsoft network protocol implementations and supported by the NTLM Security Support Provider ("NTLMSSP"). NTLM employs a challenge-response mechanism for authentication, in which clients are able to prove their identities without sending a password to the server. It consists of three messages, commonly referred to as Type 1 (negotiation), Type 2 (challenge) and Type 3 (authentication). It basically works like this:
-
The client sends a Type 1 message to the server. This primarily contains a list of features supported by the client and requested of the server.
-
The server responds with a Type 2 message. This contains a list of features supported and agreed upon by the server. Most importantly, however, it contains a challenge generated by the server.
-
The client replies to the challenge with a Type 3 message. This contains several pieces of information about the client, including the domain and username of the client user. It also contains one or more responses to the Type 2 challenge. The responses in the Type 3 message are the most critical piece, as they prove to the server that the client user has knowledge of the account password.
The following figure shows the protocol sequence between the client and server. The protocol is initiated by the client by creating a negotiate message. The Negotiate message contains the SSP signature, the message type, and the negotiate flags. The server responds to the Negotiate message by creating a challenge message. The challenge message contains the SSP signature, the message type, the Negotiate flags, and a challenge. The client takes the challenge and produces a response. It then creates an authenticate message that contains the SSP signature, the message type, the challenge response, the domain name, the user name, and the optional workstation name.
protocol Sequence between Client & Server
Proposed Scheme
The proposed scheme consists of the installation of Linux server (Fedora, Redhat, SuSE etc..) which will be easily integrated with the NT domain. Squid (http://www.squid-cache.org) will be installed on the Linux server acting as proxy and Samba with Winbind (http://www.samba.org) in order to interact with the NT domain. For further control over the access to unnecessary sites (prone sites), Squid Guard and BerkeleyDB will be installed.
Samba provides a service called Winbind, used to obtain users and groups data from an NT domain. Additionally, it provides the necessary libraries to interact with NT domain functionalities, giving access to services like Squid. Using the advantages provided by Winbind, the user authentication process will be carried out through the habitual domain controllers. This is an advantage for the users, because they will not have to remember multiple passwords.
SQUID security mechanisms includes access control lists (acl) which allow control over accesed sites. The Internet access bandwidth control, we use a mechanism offered by SQUID, taking a better advantage of the network resources. The necessary steps for the implementation of the proposed scheme are detailed in the following sections.
The blacklisted URLs are stored on the BerkeleyDB to be filtered. The URL request from client will be redirected through Squid to SquidGuard which allow or block the URL by matching over the DB.
Samba
SQUID security mechanisms includes access control lists (acl) which allow control over accesed sites. The Internet access bandwidth control, we use a mechanism offered by SQUID, taking a better advantage of the network resources. The necessary steps for the implementation of the proposed scheme are detailed in the following sections.
Installation
The last version of Samba can be obtained in the download section of the official website (http://www.samba.org/). After obtainning the desired version, we must decompress the source code for its compilation.
Configuration
After the compilation and installation processes have successfully completed, it is necessary to configure Samba and then start the daemons. We must edit the smb.conf file, located at the path specified at compilation time. The minimum necessary parameters to achieve the desired scheme are detailed in the following picture:
We compile Samba so that it behaves like a server in the EXAMPLE domain and access the Primary Domain Corntroller (PDC) to obtain users and groups. The parameters in the second section of the configuration file are necessary for correct execution of Winbind.
Using Samba
We must start two daemons, smbd and nmbd. They are UNIX® applications executed in background. Before execution of SAMBA, it is necessary to join this machine to NT Domain.
Once these three daemons are running, some tests should be carried out in order to verify Samba correct operation. We can use the wbinfo command to obtain the list of server domain users or groups.
Name Service Switch
Name Servicie Switch, also known as NSS, is a library that behaves like an interface between applications and different remote or local storage mechanisms. It is present in several systems derived from UNIX® and was developed for the GNU version of C library. The nss service allows name resolution using several mechanisms, for example nis, mysql, DNS servers, LDAP dierctory, etc.
Installation
The basic libraries of Name Service Switch are also installed with the glibc package. For additional support mechanisms, specific packages for each extra library should be installed. For example Samba should be installed in order to access domain servers data through Winbind. The libraries are installed under the /lib/ directory, their names begin with the "libnss_" string and generally end with the name of the mechanism that they support, for example libnss_dns.so, libnss_nis.so, etc.
Configuration
The /etc/nsswitch.conf configuration file should be modified in order to specify the information sources for each one of the databases and in which order the queries are made. The passwd and group databases should be added in order to integrate Winbind in the system authentication process.
Tests
Once changes have been made, we must verify the correct integration between Winbind and Name Service Switch. This can be made using the command line getent utility. Executing this command we can see the list of users or groups just like with the wbinfo command.
Squid: Optimising Web Delivery
Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages. Squid has extensive access controls and makes a great server accelerator. It runs on Unix and Windows and is licensed under the GNU GPL.
The latest version of Squid can be obtained at the download section of the official website http://www.squid-cache.org/. After downloading the desired version, the source code must be decompressed and compiled.
The necessary options to compile Squid are detailed below. The option --enable-auth="ntlm,basic" indicates that the source code must be compiled with support for basic authentication methods based on NTLM mechanism. The last one will allow the authentication of users connected to the NT domain, so they will not have to enter their password again to use Internet Explorer. If they use another browser or operating system they will have to enter domain user and password and then the mechanism used by Squid for the authentication is basic. The option --enable-External-acl-helpers="wbinfo-group" indicates wich modules are necessary to
compile in order to help Squid in the authentication process. In this case we specify winfo-group to be able to use the NT domain users groups in the access control rules. In order to carry out the basic authentication process through NTLM, it is recommended to use ntlm_auth application installed with Samba, instead of the modules that can be compiled and installed with Squid. The option --enable-delay-pools enables a granular control of the Internet access bandwidth.
Installation tests
After the correct compilation and installation of Squid, we recommend to carry out some tests before continuing with the configuration process. You have to verify the correct operation of the module that will be used in the user authentication process in Squid, as it is indicated below.
If you wish to learn more about the NTLM protocol and the interaction between the client, Squid server and PDC server in the authentication process, the information is available in: http://devel.squid-cache.org/ntlm/squid_helper_protocol.html
Configuration
In order that other applications have access to Winbind functionalities permissions of the "winbindd_privileged" directory must be changed. There is a pipe within this directory that Samba uses as interface with Winbind. The next step is the Squid configuration. You will have to modify the "squid.conf" file. In this document we only specify the necessary options to achieve the proposed scheme. You can find more specifications, parametrizations and options of Squid in http://www.squidcache.org/doc/faq/faq.html. The necessary parameters to customize the "squid.conf" file to obtain the correct integration with Winbind and to take control of the Internet access bandwidth are specified next.
#chmod 750 /var/lib/samba/winbindd_privileged
Authentication
You have to specify two mechanisms in order to make the authentication through the domain, the same ones that were indicated for the compilation process. In the first case we indicate Squid that if the browser supports NTLM it should use it, otherwise the authentication mechanism will be basic.
Access Control Lists
After specification of the authentication mechanisms, access control lists (acl) that will be used to restrict the Internet access or to control the connection bandwidth must be created. For example, we present two access control lists: the first and second lists are used to deny or to allow access to the users and the other ones to restrict the bandwidth for each workstation that accesses Internet using Squid.
Once the access control lists are defined the actions to be taken in each case must be specified. As it is detailed next, the access to the userdeny acl is denied and the access is granted to the users who enter user/password correctly.
Delay Pools
To carry out the bandwidth control Squid uses connections pools named "delay pools". According to the features of each pool, you can have one or several buckets. A bucket is a buffer that is filled or drained according to the specified filling speed and the user data consumption speed. The types of pools that can be used and their features are:
Class 1: The access of the users is limited by a single general bucket.
Class 2: The access is limited by a general bucket and a bucket for each workstation that accesses
from a class "C" IPv4 network.
Class 3: The access is limited by a general bucket, a bucket for each one of the class "B" IPv4
networks and a bucket for each workstation of each one of the networks. You have to specify in the configuration file the amount of pools of connections that are desired to be created with the delay_pools option (in this case we create a pool for each available class) and you also have to specify to which class each one belongs with the delay_class option.
Next, you have to indicate the evaluation order for the access control lists and to which pool you want to assign them.
The last thing to be done is the configuration of each pool according to the class they belong. The first parameter is the pool number and the others are a pair of values that indicates the maximum velocity that each bucket can achieve (bytes/seg) and the maximum amount of bytes that can fit in the bucket at the same time; that is the maximum size of the buffer. If it is desired that a particular bucket have no limit, you should put -1/-1 in the corresponding position.
SquidGuard
SquidGuard is an external "redirect program" whereby squid actually forwards the requests sent to itself to the external SquidGuard daemon. SquidGuard's job is to allow a greater control of filtering than Squid itself does. Although, it should be pointed out that to carry out filtering, the use of SquidGuard is not necessary for simple filters. Before compiling squidGuard, we need database engine.
Oracle BerkeleyDB
Oracle Berkeley DB is the industry-leading open source, embeddable database engine that provides developers with fast, reliable, local persistence with zero administration. Oracle Berkeley DB is a library that links directly into your application. Your application makes simple function calls, rather than sending messages to a remote server, eliminating the performance penalty of client-server architectures. Oracle Berkeley DB eliminates the overhead of SQL query processing, enabling applications with predictable access patterns to run faster.
Berkeley DB delivers the same robust data storage features as traditional, relational database systems, such as ACID transactions and recovery; locking, multiple processes and multi-threading for high concurrency; hot and cold backup; and single-master replication for high availability applications. Berkeley DB can manage databases in memory, on disk or both. Berkeley DB is designed to run in a completely unattended fashion, so all runtime administration is programmatically controlled by the application, not by a human administrator. It has been designed to be simple, fast, small and reliable.
Berkeley DB supports replication over multiple systems, enabling applications to scale massively with low latency and provide fault tolerance for high availability solutions. This technique works by having all updates go to a designated master, which distributes changes automatically to a set of replicas. The read workload can be spread across the replicas, and new replicas can join the group at any time to scale the system. If any replica fails, the remaining replicas can take over for it. If the master fails, the application can call for an election or simply designate a new master. Once the new master has been chosen, all of the replicas synchronize with the new master and move forward with normal processing with no interruption in service.
Berkeley DB is very flexible and puts developers in control of many aspects of its behavior, which allows it to be used across a wide range of applications and as a replacement for custom, home-grown solutions. For example, developers can control how resources are allocated, the amount of memory dedicated to caching records, the on-disk storage structure used for individual tables, durability and isolation guarantees, and replication policies. It includes full source code for easier porting, integration, debugging and optimization.
Berkeley DB provides very fast, reliable and scalable persistence for applications that need to store data locally, run unattended without interruption, and access data in a predictable fashion.
BerkeleyDB Installation:
Installing the blacklists
Download the blacklist.tar.gz file from internet sites
SquidGuard Installation
SquidGuard Configuration
Once SquidGuard is successfully installed, you want to configure the software according to your needs. A sample configuration has been installed in the default directory /usr/local/squidGuard (or whatever directory you pointed you intallation to).Below you find three examples for the basic configuration of SquidGuard Before we can actually start tweaking the main "/etc/squidguard.conf", we must first make one small change to our old friend "squid.conf". In the file, locate the TAG:
url_rewrite_program /usr/local/bin/squidGuard
squid -k reconfigure (to reload squid configuration)
Having told squid that we will be using a redirect program to filter requests sent to it, we must now define rules to match that.
SquidGuard's main configuration file is "/usr/local/squidGuard/squidGuard.conf". Out of the box, this file looks like the following:
Initializing the blacklists
Before you start up your squidGuard you should initialize the blacklists i.e. convert them from the textfiles to db files. Using the db format will speed up the checking and blocking. The initialization is performed by the following command:
#squidGuard -C all
References:
www.squid-cache.org
www.squidguard.org
http://www.samba.org