When we’re not working on new features, we’re often working on improving PaperCut to ensure it works well in a large variety of situations.  Sometimes customers use PaperCut in ways we never intended, or encounter scenarios which we never envisioned.  We take pride in listening to our customer experiences, and making improvements to ensure we meet their needs.  The result is a better product, and everyone benefits.

Today I’ll explain an issue raised recently by one of our largest PaperCut ChargeBack customers, and then describe how we designed and implemented the solution.

The Problem

This customer was running PaperCut in a challenging environment:

• Around 2,000 users all running the user client to charge printing to shared accounts;
• Users are located at a number of remote sites over slower network connections;
• Printing is charged to over 30,000 shared accounts;
• All users could charge printing to any of the 30,000 accounts;
• Each night an automated process added, removed or modified hundreds of accounts.

They reported the following problems:

• The user clients on remote sites were slow to popup when first printing.
• A large amount of bandwidth was used communicating with the primary PaperCut server.
• The PaperCut server was slow to respond in the mornings.

Analysis

It was clear that the cause of all these symptoms was the large amount of account data downloaded by each of the user clients.   The user client will only download the account list from the server when it detects accounts have changed.  But at this customer the account list was changed nightly.  This caused the client software to re-download the large account list when the user first printed each day.

The client software communicates with the server using XML web services.  After some analysis we found that the the XML representation of the 30,000 accounts was about 7.5 MB.  On a fast local network this size is not a problem, but with 100’s of clients running on remote sites over slower WAN connections this starts to be an issue.  It causes congestion on the network, the popup is slow to appear, and the server becomes overloaded.

After a few brainstorming sessions and consulting with the customer we approached the solution from two angles.

1. Reduce the size of the account download;
2. Allow the account data to be efficiently distributed to remote sites.

Solution Part 1: Reducing the download size

To reduce the size of the data downloaded we changed the account file format in two ways:

1. Change from verbose XML to a much more efficient binary format;
2. Compress the binary data

The net result was a reduction of approximately 25 times!   In the case of this customer the 7.5MB download was reduced to around 250KB. Quite an improvement!

This change alone dramatically reduced the popup speed, and bandwidth used over the WAN connections.  But we could do better!

Solution Part 2: Efficient distribution

We realized that it was very inefficient for multiple clients at the same remote site to download the identical account list.  What if we could only download the list once per site?

We implemented a server-side tool to save the account list to a file (in the identical format described above).  We added a new option to the client software to allow it to load the account list from a file instead of downloading from the server directly.  Then all that this customer need to do was:

1. Schedule a task to create the account file each night (after the accounts were modified), then copy the account file to a file share on each of the remote sites.
2. Configure the clients at each site to load the local account list file.

The result was that only a single 250KB transfer was required to delivery the new account list to all clients at a given remote site.

Conclusion

The result of these changes was dramatic.  Previously, on a site with 20 users the daily account download would consume around 150MB of bandwidth.  After these changes it was reduced to only 250KB.  What an improvement!

Not only was this a great result for this customer, but the product is improved for everyone.

I hope this gave you a little insight into the work we do behind the scenes.  This change wasn’t glamorous or exciting.  It will only be a single bullet point in our version 8.4 release notes.    However it’s often many of these small improvements that make the overall product great.

We often get requests from customers that want to be notified when we release new versions. They want to run the latest version to take advantage of all the new features (without having to check our web site every week).

So today I added news feeds to the website to notify you as soon as a release is available. There are news feeds for both PaperCut NG and PaperCut ChargeBack and are available from the release history and download pages. For the impatient, here are the feed links:

• PaperCut NG Releases
• PaperCut ChargeBack Releases

News feeds are a great way to keep up-to-date with blogs and news. To subscribe to a news feed you need to use a news reader (or aggregator) which is a piece of software (or web site) that retrieves news items from all sites your are subscribed to and notifies you when you have unread items.

Both Firefox 2 and Internet Explorer 7 have built-in feed readers, but they are pretty basic. If you’re interested in something more powerful we highly recommend Google Reader. All the developers at PaperCut use Google Reader daily to keep up-to-date with what’s new in the world.

Happy news reading!

I’ve just uploaded another build of the Net Control module. This release includes a number of changes:

• Add “enable/disable user net usage” option to the bulk user action page, so this setting can be set on users in bulk.
• Add support for the MS ISA Server log format, in addition to the preferred W3C log format.
• Improved detection of the latest log file based on the specified log file mask.
• Improved behaviour when processing MS ISA log format, when the W3C log format is specified in PaperCut.
• Do not log net usage performed by Squid users that haven’t authenticated correctly.
• Fix to error that occurred when running a transaction log report that contained Internet usage transactions.
• Fix bug in Linux installer that caused the Internet service config file to be overwritten when upgrading from a previous version.
• Improved the setup of the Internet Provider service on linux to warn the user if the specified log file directory is not readable.
• Add a config option to enable LDAP referrals.
• All the many changes included in the most recent public release of PaperCut NG 6.3. For more information see the release history page.
• Other small fixes.

You can download the latest release (Build 4006) from the Net Control module testing page. To upgrade from a previous release you can perform an install over the top procedure.

I’ve just uploaded another build of the Net Control module. This release includes the following changes:

• All the changes included in the most recent public release of PaperCut NG 6.2. For more information see the release history page.
• Other small fixes.

You can download the latest release (Build 3909) from the Net Control module testing page. To upgrade from a previous release you can perform an install over the top procedure.

I’ve just uploaded another build with a few small fixes and changes, including:

• Allow the internal admin password to be changed from within the user interface (on the Options->Advanced) tab.
• In the Internet Usage logs screens, allow the results to be sorted by the “MB received (ex cached)” column.
• Fix error that can occur when saving groups if only the Internet Control module is enabled.
• Other small fixes.

You can download the latest release (Build 3858) from the Net Control module testing page.

Just touching base to let you know that we’ve released a new build that includes a number of changes (a number as a result of feedback from our testers). The testing has been going well. We’ve had no reports of any major problems. Most of the comments have been about feature suggestions or usability suggestions.

You can download the latest release (Build 3853) from the Net Control module testing page.
The latest build include the following changes:

• Allow the Internet usage for a single user to be disabled for a specific amount of time or indefinitely (time latches). This can be useful to discipline students who have abused Internet privileges, but you still want them to be able to print.
• Allow data costs to be defined “per gigabyte” in addition to “per megabyte”
• Allow the user client window position to be changed. This allows multiple user clients to be run when different PaperCut instances host Net and Print control.
• Clarify some of the terminology to make it easier to understand whether data includes cached data or not.
• Ignore machine accounts (e.g. username$) in proxy logs. These entries are recorded when a Windows machine downloads Windows updates.
• Improved LDAP support for large sites.
• Licensing changes to allow users who are registered for Print Control to be able to trial the Net Module for 40 days.
• Bumped the version number to 7.0.
• All the updates included in the latest 6.2 release (see here for full details).

We just released an updated Windows build of the Net Control release that includes a fix for using domain groups when PaperCut is running on a member server. The problem was that PaperCut was only listing local groups in the setup wizard, so domain groups could not be used. If you were running PaperCut on a domain controller then the domain groups will list correctly, and there is no need to update.

You can download the latest release (Build 3816) from the Net Control module testing page.

Wow, it’s been a very long time since my last post… I am alive!

We’ve been really busy over the past few months making PaperCut NG even better, and just released version 6.1. It adds a number of often requested features like LDAP, but some of the best improvements have been under the covers in the User Client, including:

• Much faster print notifications
• Much improved scalability when running thousands of clients simultaneously
• Reduced bandwidth usage
• More robust when the client can’t connect to the server upon startup

We had a client running the User Client on almost 2000 workstations simultaneously!! And this was putting considerable load on the PaperCut Application Server, so we had to rethink our approach. The result was a large re-write of the client and the code on the server that handle client connections. If you’re a software developer and you’ve ever ever had to do this type of optimization, you’ll know that it’s quite tricky. You’ll also know that the bottlenecks are often not where you thought they would be.

But the effort was worth it!

The client now consumes very little bandwith (10% of the bandwith it did previously), the print notifications are almost instantaneous, and the server handles thousands of simultaneous connections without breaking a sweat.

… anyway … back to work!

It’s been a while since my last blog … so I thought I’d make do an update on my series about configuring Squid proxy.

This time I’m going to concentrate on SquidNT, which is a version of Squid compiled for Windows, that come with some useful helpers to allow authentication with Windows and Active Directory.

The key advantages of SquidNT, when compared to Squid on Linux/Unix are:

• A bit easier to get going if you’re not familiar with Linux/Unix
• You can configure Windows authentication more simply, and your users will be authenticated automatically without being prompted for their username/password.
• Don’t need to know how to install/compile Unix applications (We’ve had customers whose Linux distribution did not include the latest version of Squid and had troubles authenticating with Windows. Getting an unsupported version of Squid onto the distribution proved difficult.)

The major problem with SquidNT, is that the installation documentation is hard to come by. Although the core Squid documentation is good, the Windows specific documentation is not easy to find… so hopefully my article will help.

So if you’re interested in running Squid on Windows, then read my article: Installing and Configuring SquidNT. If you have any questions, then feel free to get in touch.

(If you want to use Squid on Linux/Unix and authenticate with Active Directory, then my previous article is the best place to start.)

There are about 250,000 members of SpreadFirefox, and we’re currently 10th in the SpreadFirefox rollcall.

It probably won’t last long … and it might not happen again … but here’s proof.