The Dev. Team here at PaperCut recently did a presentation about Apache Derby at the Australian Java User’s Group (AJUG) meeting in Melbourne. Apache Derby is the default database option supplied with PaperCut. Even thought we offer a choice of MS SQL Server, Oracle and Postgres within PaperCut, we find that 90% of our 10,000+ users stick with Derby. It’s a great choice for most of the medium to smaller sizes organizations as it’s a self-managing database. This means that you as an administrator don’t have to get involved with traditional DBA management tasks such as off-line backups, indexing, and performance tuning. The application and embedded database handles this all for you!

Many will have heard about databases like Oracle or MySQL. Apache Derby however is a little more discreet. The reason for this is that it’s an embedded database. That is, it’s designed to be pre-packaged with applications as a library rather than being deployed as a separate standalone component. However just because it’s has a “low profile” doesn’t mean its not good. It our opinion it’s one of the best databases around. Its performant, packaged full of features, and has a fantastic pedigree being born out of the database development teams at IBM, Informix and Sun. Apache Derby was open-sourced in 2005 when IBM donated it to the Apache Foundation. In now continues to be activly developed under the stewardship of the Apache Foundation along with many other world-class projects such as the Apache HTTP server.

If any are interested in knowing more about Apache Derby and its behind the scenes use in our print monitor application then please check out the slides from the presentation (PDF or HTML).

The internet is a great place to laugh and poke fun at the grammatical abilities of its denizens. Not that I claim to be such an expert; I just enjoy the occasional entertainment at someone else’s expense. While some, like “you loose” and “your a …” are just plain annoying, there is the odd gem like cereal killer (that link is from FAIL Blog’s Burn of the Week, which has been a bit of a regular for classic grammar blunders).

Fortunately for the grammatically challenged, there are a few factors that save from too much embarrassment:

1. A spell checker
2. Anonymity
3. Obscurity (the fact that very few people are actually going to read what you wrote)

I’d like to highlight an example that failed all three, is possibly the most common misspelling of all time, yet is one I’d never seen or heard of until the other day.

I was recently working on integrating PaperCut’s Payment Gateway Module with the Barclaycard ePDQ CPI service, allowing students (or others) to transfer value from their bank or credit account into their PaperCut account, which they can then use in addition to any print quotas they receive.

One of the security features of the Barclaycard service is to only accept connections from users who have been redirected from a particular URI. This is a feature of HTTP called the referrer URI - our web browsers tell the web page we are visiting where we just came from. This information is valuable for web masters to understand where their viewers are coming from. In this case, only accepting users who have come from a particular URI prevents a malicious site trying to make use of the service.

During development, this turned out to be more of an annoyance than anything. Luckily, there is an easy way to set/fake the HTTP referrer URI in Java:
urlConn.setRequestProperty("Referrer", "http://my.allowed.url/");

Something was wrong though… the above line didn’t seem to be working, or at least Barclaycard was still refusing my connection. Perhaps I spelt it wrong? A quick search in an online dictionary confirmed I’d spelt it correctly. But wait, what was the next entry?

referer

A misspelling of “referrer” which somehow made it into the HTTP standard. A given web page’s referer (sic) is the URL of whatever web page contains the link that the user followed to the current page. Most browsers pass this information as part of a request.

(from the Free On-Line Dictionary of Computing)

Well that was a surprise. Not only did someone misspell the word when formulating the standards document and possibly fail to run a spell checker, but supposedly there was no-one amongst the committee or technical consults that successfully recommended a correction. What a colossal screw up. Even the official HTTP/1.1 specification has this to say:

The Referer[sic] request-header field allows the client to for the server’s benefit, the address (URI) of the resource which the Request-URI was obtained (the “referrer”, although header field is misspelled.)

So, given that HTTP referrer URI is used every time any person clicks a link on the internet, I’ll vote it the most common misspelling of all time.

On the flip side, as Chris pointed out, this misspelling has saved the world a whole heap of bandwidth: 1 byte for every link ever clicked!

My first son William arrived yesterday. Unlike our planning here at PaperCut which usually run a few weeks behind schedule, my first child has decided to buck the trend and run ahead of schedule. William weighed in at 1.795kb (just under 4 pounds) - very small but strong and healthy. Both Mother and baby are doing well and Dad is very proud.

William’s bound to be a “tech baby” and I’m sure will be writing computer code in a few years. I was meant to be presenting at the Australian Java User’s Group the night he arrived. He was obviously very keen to get out and attend (also thanks to Matt who stepped in and did the presentation for me). He’s first taste of technology was his photo being sent out to everyone a few minutes after his birth courtesy of a 3G iPhone!

I’m sure to have my hands full over the next months. Lots of time looking after baby and working on print control software between feeds!

Another update for the sites participating in the Novell OES Linux beta program. It looks like we’ll have to wait a few more weeks to get proper print quota on iPrint. The iPrint update required to run PaperCut will now not be available until next week. Novell will be undertaking the last validation tests on Friday and assuming they all pass, they’ll be looking at supplying us with the update on Monday. The build is going to be based on the iPrint Beta 4 release. I’ll keep you all posted and let you know if there are any further delays.

Last week we announced the start of the official beta testing period for PaperCut on Novell. Novell is our last remaining platform so it’s great to see this finally in beta. We already have close to 50 sites that have contacted us expressing eagerness to participate in the beta testing program. There is a lot of excitement out there and a lost of organizations are happy to finally see best of breed print auditing and print control on iPrint.

As discussed in this page, I am going to post updates, news and announcements in our developer blog. One of the immediate announcements is with respect to the availability of the new iPrint release from Novell. PaperCut targets a yet-to-be-released iPrint version. This is a new version of iPrint scheduled for official release in Novell OES Linux 2 SP1 and includes the API hooks we required to get PaperCut working and cooperating with iPrint. Dean and Devon at Novell are working through the internal procedures at the moment so all beta test sites have early access to this iPrint update. The plan was to have this available last week but it’s been delayed. Dean is aiming to have all approved and available on Thursday the 21st.

In the meantime please regularly check this Blog for any news. Also if you have any questions or what a sneak peek, please check out the announcement and/or feel free to email me at support.

As keen Java programmers we’re always on the lookout for interesting tidbits about the language, how it is being used and where it is heading. Just the other day Matt brought in a copy of Java Puzzlers to test our knowledge. So it was something of a coincidence that the guys over at Atlassian came out with their own Java puzzler.

The puzzle is a Java version of the shell game (AKA ‘thimblerig’), where the onlooker must choose which shell the ball is under (after the con artist has shuffled them around).

The puzzle and the included solutions are interesting, and well worth a read if you’re a programmer. Chris came up with the ’static block’ method almost immediately, but I must admit that my thinking was just along the lines of “err, with reflection somehow?” before we looked up the answers.

Four solutions were presented on Atlassian’s blog post, but I have a fifth: The terms of engagement were “you can only modify the class representing the ball”. But what about a solution that doesn’t involve modifying the ball either? What if you could alter the onlookers’ perception so that when the cup is raised they see a ball, despite there not being a ball at all?

Using the Force and with a wave of the hand, we get:

aspect YouWillBeEqual {
    boolean around(): call(boolean String.equals(Object)) {
        return true;
    }
}

Now there is a ball under every cup, always. At least, it appears that way. Actually, everything around here appears to have a ball under it…

Made possible with a little AOP

When we’re not working on new features, we’re often working on improving PaperCut to ensure it works well in a large variety of situations.  Sometimes customers use PaperCut in ways we never intended, or encounter scenarios which we never envisioned.  We take pride in listening to our customer experiences, and making improvements to ensure we meet their needs.  The result is a better product, and everyone benefits.

Today I’ll explain an issue raised recently by one of our largest PaperCut ChargeBack customers, and then describe how we designed and implemented the solution.

The Problem

This customer was running PaperCut in a challenging environment:

• Around 2,000 users all running the user client to charge printing to shared accounts;
• Users are located at a number of remote sites over slower network connections;
• Printing is charged to over 30,000 shared accounts;
• All users could charge printing to any of the 30,000 accounts;
• Each night an automated process added, removed or modified hundreds of accounts.

They reported the following problems:

• The user clients on remote sites were slow to popup when first printing.
• A large amount of bandwidth was used communicating with the primary PaperCut server.
• The PaperCut server was slow to respond in the mornings.

Analysis

It was clear that the cause of all these symptoms was the large amount of account data downloaded by each of the user clients.   The user client will only download the account list from the server when it detects accounts have changed.  But at this customer the account list was changed nightly.  This caused the client software to re-download the large account list when the user first printed each day.

The client software communicates with the server using XML web services.  After some analysis we found that the the XML representation of the 30,000 accounts was about 7.5 MB.  On a fast local network this size is not a problem, but with 100’s of clients running on remote sites over slower WAN connections this starts to be an issue.  It causes congestion on the network, the popup is slow to appear, and the server becomes overloaded.

After a few brainstorming sessions and consulting with the customer we approached the solution from two angles.

1. Reduce the size of the account download;
2. Allow the account data to be efficiently distributed to remote sites.

Solution Part 1: Reducing the download size

To reduce the size of the data downloaded we changed the account file format in two ways:

1. Change from verbose XML to a much more efficient binary format;
2. Compress the binary data

The net result was a reduction of approximately 25 times!   In the case of this customer the 7.5MB download was reduced to around 250KB. Quite an improvement!

This change alone dramatically reduced the popup speed, and bandwidth used over the WAN connections.  But we could do better!

Solution Part 2: Efficient distribution

We realized that it was very inefficient for multiple clients at the same remote site to download the identical account list.  What if we could only download the list once per site?

We implemented a server-side tool to save the account list to a file (in the identical format described above).  We added a new option to the client software to allow it to load the account list from a file instead of downloading from the server directly.  Then all that this customer need to do was:

1. Schedule a task to create the account file each night (after the accounts were modified), then copy the account file to a file share on each of the remote sites.
2. Configure the clients at each site to load the local account list file.

The result was that only a single 250KB transfer was required to delivery the new account list to all clients at a given remote site.

Conclusion

The result of these changes was dramatic.  Previously, on a site with 20 users the daily account download would consume around 150MB of bandwidth.  After these changes it was reduced to only 250KB.  What an improvement!

Not only was this a great result for this customer, but the product is improved for everyone.

I hope this gave you a little insight into the work we do behind the scenes.  This change wasn’t glamorous or exciting.  It will only be a single bullet point in our version 8.4 release notes.    However it’s often many of these small improvements that make the overall product great.

Many companies have a Jobs/Employment section on their website. We’re a small company with very low staff turn over and hence rarely advertise positions. We do however have one going right now! Quite a few people subscribe to our blog, so I thought we’d do some shameless self promotion and post the position here:

Read the rest of this entry »

Since we release our free print logging program a few years back its been a hit with hundreds of thousands of downloads. Every now and then we receive a thank you email or a short story about how people are finding it useful. One user recently wrote to us to thank us for helping him track down why his ink was disappearing. He had no explanation as to why his home printer ran out of ink every few weeks. Print Logger to the rescue! A quick audit of print activity shows that his kids did a lot of printing before he got home from work. Now printing in his household is restricted to homework use only and he’s happy again!

There are also plenty of other examples at the big end of town. Today I received an email from PepsiCo and HP outlining their use of PaperCut Print Logger. It’s great to see that this little free program has uses ranging from small homes to the largest corporates.

For us, PaperCut Print Logger serves two purposes:

1) It’s a great test bed for our printer page analysis technology. With hundreds of thousands of users, we quickly get reports about incompatibility with new drivers. The program actively encourages users to report incompatibilities and in turn this ensures that all our applications offer the widest range of support.

2) It’s also a fantastic way to get a “taste” of our applications. Many schools for example will install PaperCut Print Logger to get a quick view of what’s going on on their networks. It’s then only a small jump to move to PaperCut to implement print control, quotas, reporting, and of course monitor the environmental impact of printing.

If you’re running print logger, please take a few moments to send us an email and share your stories.

The other day I sent a quick email to the Jetty HTTP Server development team thanking them for all their hard work. We struck up a bit of a conversation and they wanted to know more about how we’re using Jetty in PaperCut and asked us if we’d like to write up a short Case Study for their website. I’ve decided to post this on the blog as it will also give some of our users an interesting look at the “internals” of PaperCut.

Our use of Jetty is quite a bit different to most. Instead of treating Jetty like Apache or a stand-alone server, we’re using Jetty as an embedded component. It’s deployed with our print accounting application, PaperCut (tour here for those interested). As a result, Jetty is running on tens-of-thousands of servers in over 60 countries and the vast majority of users are completely unaware that it’s there. The measure of a good embeddable component is not who’s using it, but who’s not aware that they’re using it! Jetty is fantastic in this regard - zero config and zero maintenance, working away flawlessly for years on end.

Jetty is hosting PaperCut’s web application interface. This is a web application based on Tapestry, Spring, Hibernate and Apache Derby. Jetty’s embeddability has allowed us to deploy this stack in a standard setup.exe type installer targeting all major platforms (Windows, Linux and Mac). Our users don’t need to know anything about web servers or their setup and configuration. All they need to do is run the installer! Jetty’s scalability has also been an advantage. In some cases PaperCut is installed on a small business server supporting 5 workstations, while on other sites it’s running in University environments on clustered servers supporting 100,000+ users. Jetty has prove equally adaptable at both ends of the scalability spectrum.

Choosing components for a project is always difficult - often too much choice! Obviously Jetty has been a great choice but was one we carefully considered back in 2005. Technical functionality was one criteria, but another important consideration was the strength of the team behind the project. We’ve seen too many Open Source projects start out with enthusiasm only to run out of puff and “age” early. What grabbed us about Jetty was the future looking team. In 2005 AJAX was very new and there was big questions about how to handle server-push events in a scalable fashion. We had an immediate need for this, not with AJAX, but with our web services API (client software receiving web-services push events from the server). The Jetty development team members such as Greg Wilkins, were openly discussing “long polling” and the issues around this. Jetty was one of the first servers to present working and more importantly “workable” solutions to the scalability problems.

We can’t recommend Jetty more highly. It’s a great web server, a great component, and is backed by a switched-on forward looking and innovative team - make sure you check out the open source Jetty and the team!