PaperCut Blog / News

42 Printer Languages and Counting

Since joining the PaperCut team, I have mainly been working on the area of PaperCut responsible for the analysis of print jobs. I have come from a background of programming in the computer language, C, doing file system programming in the Linux kernel. I worked for SGI on the XFS file system team. The job analysis code in PaperCut is entirely written in C which has given me a chance to sink my teeth into something quickly. Luckily for me it is also well written C code and has been designed to be easily extended for new print languages and print drivers.  Every few weeks printer manufactures release new print drivers and these need to be tested with PaperCut.  Changes are always occurring.  Sometimes these changes are minor and just a little bit of tweaking is required, while others are a lot more complex. In some cases we need to support whole new print languages.

Many of the lower cost printers these days are using GDI drivers.  These are drivers that don’t support known popular standards like PCL or PostScript and implement their own protocols – often not documented nor following any existing standard. In order to handle new GDI printer languages, it’s a bit like playing a game of detectives. By reverse engineering, we try out various sample documents, changing different page attributes and work out how the driver encodes the various attributes (e.g. duplex, gray-scale and paper-size). We have a set of pattern tools and a few techniques that help us perform this decoding process.  After a bit of teeth pulling we develop an algorithm to support the new driver.  The techniques are in some ways similar to those implied in reverse engineering file system formats!

After we’ve made changes to our print analysis code, we run our analyzer tool over a corpus of over thousands of real-world documents to ensure we haven’t introduced any regressions (steps backward).  The tools also check for memory leaks and test performance against previous test runs to make sure the software is not getting slower.

I hope you’ve enjoyed the behind the scenes look into some of the technical workings here at PaperCut. For a less technical look, make sure you check out what print management has to do with Coffee?

42 Image from Answer To Life / CC-BY-SA

A Printing Press

When counted by number of pages printed, most printing still takes place on printing presses like the one on the right.

The following table and pie chart, based on numbers from an American Printer article, show the state of the US printing industry in 2004.

Printing Industry by Segment

Within this slow-moving industry there have been dynamic pockets of rapid growth and the adoption of modern technologies for the last 25 years. One of these has been digital printing.

Chester Carlson

My employer was one of many small companies thriving in the market opened by the cheap Apple LaserWriter PostScript laser printer in 1985 and the cheap desktop publishing applications created by Apple, Adobe and other development companies. This market remains vibrant as work that was previously done in other parts of the printing industry moves to digital color printing and more corporate printing moves in house as office printers and office software becomes more capable. (There was a parallel and related trend of growth built around home computer printing and drop on demand (dod) inkjet technology but this post will focus on graphics and office printing.) A brief history of digital printing follows.
(more…)

I was recently helping a customer with a couple of reporting questions they had and they mentioned that they were going to be rolling out 20 new printers in the near future. I commented that this would require quite a bit of work, lugging hardware around, changing printer drivers etc. They agreed and then said that the worst of it was setting the static IP addresses of all of the new devices.

Having had to manage rollouts of small to large sizes I’ve come across this problem before and luckily found two features of a Microsoft DHCP server that can save a lot of time.

DHCP Reservations

Ultimately there is no difference between a static IP address and a DHCP allocated IP address that doesn’t change. The results are the same, you connect to the IP address and the same device is there each and every time. This is essential for servers, switches, routers, printers and more. In many cases setting the device to use a static IP address is suitable and possibly even best practice (routers in large networks for example).

However, if you have a large number of devices that require non-changing IP addresses, why not set your DHCP server to always give out the same IP address each time the device turns on or refreshes its IP address? Simply open your DHCP management interface (I’ll assume Windows here) and navigate to your Scope and then Reservations. From here you can create new reservations and all you will need is the MAC address from the printer. For example: 00:1D:09:FE:64:04 is converted to 001D09FE6404 and given then IP address of 10.1.1.12. Set the printer to DHCP assigned address and give it a moment.

A couple of seconds later you should be able to communicate with the printer on the new IP address.

The benefits of this are quite easy to see, imagine if you had to send a printer to a remote office where there are no tech-savvy users. Grab the MAC address of the printer, create a new DHCP reservation, create the new print queue and share it out. When the printer gets out there, simply get someone to plug it in to power and network and you’re done.

If you replace the device (hardware does fail unfortunately), simply change the MAC address in the reservation then replace the old hardware. No fiddling with the interface panel on the printer required.

DHCP import/export

“That’s nice.” I hear you say, “But it doesn’t help with these 20 devices, I’m at the printer anyway, might as well do the time there.”

This is quite valid, except you can export/import DHCP settings via the command line enabling you to use your favourite spreadsheet program to manipulate the data.

Running the following command will give you a simple dump of the existing DHCP setup.

netsh dhcp server 192.168.0.1 dump > C:\Dhcp\Dhcpcfg.dmp

Inside you will see commands like:

dhcp server \\10.1.1.1 scope 10.1.1.0 add reservedip 10.1.1.12 001D09FE6404 "printer-library" "" "BOTH"

By adding “netsh ” to the start of those lines, you have a pre formatted script to add DHCP configuration settings.

Congratulations! You’ve now set a reserved IP.

If you wanted to get creative, you could export your existing DHCP reservations, load the file into Microsoft Excel, modify the MAC addresses for the relevant devices and re-import it to your DHCP server. 30 seconds per printer by this method, or several minutes standing at the print queue.

Remember, this is just for printing, there is no reason you can’t use this for new servers, printers, switches, access points, specialized desktops and more.

For more netsh DHCP commands see Microsoft’s TechNet website:
http://technet.microsoft.com/en-us/library/cc787375(WS.10).aspx

For more information in general about netsh commands see Microsoft’s Help and Support website: http://support.microsoft.com/kb/242468

Hope this post is useful. Best-practice printer management on a large sites not just about PaperCut. It’s also about your network management practices!

I recently helped out one of our biggest corporate customers to resolve issues with their print server. During the the last week of the financial year (when printing load is the highest) their print server became overloaded and stopped working. This sounds bad, but we quickly got things working smoothly again and learned that …

This customer had been running PaperCut for about 6 months without issue. Over this period they were gradually transitioning 100s of print queues from legacy print servers to the server hosting PaperCut. This single print server was hosting all queues for their offices country-wide. The extra load of these additional print queues combined with the end-of-year printing load pushed the server to the limit.

When analyzing the problem I noticed that this server was handling a huge print load. In the 30 day period prior the following printing occurred:

• 477,287 print jobs
• 2,021,454 pages printed
• Between 22,000 to 25,000 print jobs each week day

Wow! That’s a lot of printing!

They were also using hold/release queues and Find-Me printing (aka follow-me printing) to provide secure print release and to reduce paper wastage. The result was an average of around 500-600 print jobs waiting in the queue to be released.

The cause of the problem was under resourcing. Their setup was:

• A single server hosting the both the print queues and the PaperCut application server
• The server was a virtual machine assigned only a single processor
• Allocated 3GB of RAM
• Running on a 32-bit Windows Server operating system

My recommendation was to leave the print queues on the existing server, but move the PaperCut Application Server service to a server with 4GB of RAM, 2 or more processors, and running a 64-bit operating system with the 64-bit add-on pack. This configuration:

• Spreads the load between 2 servers
• Allows the PaperCut Application Server to take advantage of more memory (64-bit)
• More available processors allowed efficient processing of simultaneous print jobs

Since making these changes, their system has been running very smoothly. Their servers are now handling more load than ever, and without overloading the servers.

If you’re managing a large PaperCut installation, and in particular leveraging some of PaperCut’s advanced print management features such as secure print release, then there’s a few lessons to take from this:

• Don’t skimp on RAM or CPU resources
• Monitor your servers. Particularly if you’re adding print queues and increasing print load
• Consider running a 64-bit OS to allow for future expansion (e.g. more memory)
• Run PaperCut on an external database like SQL Server, Oracle, PostgreSQL or MySQL

CC image courtesy of Emilian Robert Vicol on flickr

Priyanka transitions from Java code to Java drink!

The regular readers of our blog will have noticed a few off-topic posts slipping in from time to time. The common theme is coffee and beer. As a group of passionate computer programmers and tech geeks it’s no surprise that we have developed a strong corporate coffee culture. Coffee is our secret weapon! Over the past 10 years we’ve changed programming languages, compilers, and development practices, but one factor has remained constant: Coffee. It must be the pillar for PaperCut’s success.

Coffee is very much part of our culture. The company funds a continuous flow of lattes, cappuccinos and macchiatos (Hendrik’s favorite) all arriving from the coffee shop directly opposite the office. Most of us have espresso machines at home (e.g. Rancilio Silva) and discussions on brewing techniques seem to pop up in developer meeting agendas unannounced.

Recently management decided that attending a formal coffee barista course would be a good idea. Traditional businesses would have called this a “cooperate team building exercise”, however for us it’s “core competency training” The whole Melbourne development team (minus Tom) spent a day at a coffee training academy learning the finer points of coffee production.

Lessons included:

• The art of wasting lots of milk perfecting the perfect froth.
• The amount of coffee one must waste to calibrate the ideal 25 second espresso pour.
• Latte art: The art of convincing someone that the shape on the top of their coffee was deliberate.
• How to make beverages unknown to computer programmers (chai lattes, and hot chocolates)

The day finished off with a competition. We paired up into teams and had to make 8 coffee variants in 8 minutes. Congratulations to Matt and Jason who took out the title.

To take a slight deviation, my favorite pieces of coffee trivia:

• The magic number on Java .class files is 0xCAFEBABE
• The embedded framework we target for our HP embedded MFP development is called Chai

Overall it was a very fun day. We even got to walk away with a formal certificate – we’re now qualified Baristas! If we all get sick of writing print management software we now at least have a fall back option – open a Cafe!

Thanks to Jason for the great images!

A few weeks ago we launched our new website design. Working on the website took up the majority of my time in the few weeks prior. As part of the redesign we added a number of new pages, removed some and moved others from one place to another. During this process a question was raised, “Should our URLs have trailing slashes?”. What we were talking about is this: http://example.org/page/ versus this: http://example.org/page . The former has a slash on the end, the latter doesn’t. Does it matter? Which one is better?

Does it matter?

It certainly matters if you are allowing both URLs to access the same content. Google’s SEO Starter Guide says

“Provide one version of a URL to reach a document”

or risk splitting the reputation of the page between all the URLs used to access it. This means that at best it won’t matter for you. At worst you’ll have your page’s reputation diminished, and you might also be penalised for duplicate content.

The best option is to pick one scheme, use that scheme in all your links, and redirect users who access your pages using the other scheme.

So it probably matters. Now do we slash or not?

I lean heavily towards slashless URLs, but at first I couldn’t put a finger on why. “They just look right”. So I thought about it for a while, made a list of the factors I could think of, and wrote it up as a blog post.

Semantics

What does it mean for a URL to contain a trailing slash? For me, it traditionally means “this is a directory listing”. When performing a request on a URL that maps to an actual directory in the file system in absence of an index document most web servers serve up a listing of the files in that directory. This is similar to doing an ls or dir from a terminal. Does the same apply when accessing http://www.papercut.com/blog/? That page is serving up the X most recent blog posts, and the view might depend on whether or not you’re logged in. In my opinion it is not a canonical list of sub-items items being served up, so the directory analogy doesn’t fully hold.

The more interesting part of how slashes affect semantics is the format of the response. What’s actually happening when a web browser requests /blog is the web server recognising that no specific format was requested in the URL (i.e. there was no .html extension), but because it’s a web browser making the request it makes an assumption that it wants the response in HTML. On the modern web that’s no longer a given. We have .json, .atom, .rss, .xml and others. If a browser wants a view of something in a format other than HTML, it makes sense to allow that simply by adding the requested extension. E.g. http://example.org/blog.atom . If the URL had a trailing slash that would become http://example.org/blog/.atom, which looks horrible.

Reddit provides at least one example of where both a trailing slash and format extension are used together!

Role Models

What are the big boys doing? Google sometimes add slashes and sometimes don’t, but they pick one and use redirects to enforce it. Stack Overflow allows both (!), but their own links omit them. Wikipedia omits slashes and doesn’t know what you mean if you add one.

Ruby on Rails put a lot of thought into their URL routing functionality. The result is, in my opinion, the most intuitive and complete definition for URL schemes anywhere. It should serve as a model to other frameworks and to those creating URL schemes by hand. Oh, and trailing slashes aren’t used (unless you go through some extra work to add them back in).

Legacy

For us the main factor was legacy. Most of our website URLs would remain the same (we were just introducing some new ones), and they already had trailing slashes. Would it hurt to redirect all the slashful URLs across to slashless ones? The best we could come up with is “maybe”, but that was enough to can the idea. Redirecting one URL to another via a 301 redirect (“moved permanently”) is rumoured to result in the page’s reputation flowing to the new URL. In practise, and confirmed at least once by Google, some of that reputation will be lost.

We’ve done a similar thing once in the past when we switched our main domain from papercut.biz to papercut.com. We used 301 redirects for all our URLs but several pages lost some reputation (e.g. from PageRank 6 to 5). Any pages that lost reputation gained it back after a month or two, however.

Implementation

Ask four web devs how to implement pretty URLs or remove/add your slashes and you’ll get seventeen answers. We use Apache with PHP, and implement the redirection in our root .htaccess file via Apache’s mod_rewrite.

Firstly we provide access to .php pages using “directory naming”:

RewriteCond %{PATH_INFO} ^/$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME}.php -f
RewriteRule . %{REQUEST_FILENAME}.php [L]

(the above reads “if the requested filename doesn’t exist as a real file or directory, but adding .php on the end results in a real file, serve up that .php file (but don’t change the URL”).

Then we add a trailing slash if none was present in the request:

RewriteCond %{PATH_INFO} ^/?$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME}.html -f [OR]
RewriteCond %{REQUEST_FILENAME}.php -f
RewriteRule (.*[^\/])$ $1/ [R=301,NS]

(the above reads “if the requested filename doesn’t exist as a real file or directory, but adding .php or .html on the end results in a real file, if they didn’t add a trailing slash then send the browser a permanent redirect to add the slash”).

Search on this topic and you’ll find hundreds of ways to do similar things, many of which have subtle problems in certain situations. Ours probably isn’t perfect, but it’s been working for us so far.

Summary

• Picking slashful versus slashless URLs probably matters. Pick one or the other, don’t allow both to return the same content.
• Accessing a URL with a slash on the end is not fully analogous with performing a directory listing.
• Slashless URLs look much better when you want to support multiple formats (/page.json not /page/.json).
• In the wild, people do it both ways.
• Ruby on Rails has a very nice and complete system for dealing with URLs, and they don’t use trailing slashes.
• If you already do it one one, your pages will probably lose some reputation if you redirect them to the other way.
• Implementation is a black art. Allow yourself time to understand the details.

In the world of software systems, blind faith is often consider the province of the naïve and inexperienced. The high potential for surprises in software deployments has given rise to the following axiom.

If you use a piece of hardware long enough, eventually it will break. If you use a piece of software long enough, eventually it will work.

This somewhat dark reputation of software has grown over decades of implementation horror stories that have a much longer lifespan than the multitude of boring stories of successful deployments. When things go bad with a software implementation the damages are often characterized as a waste of time, money and productivity; the very areas that the software was supposed to improve! This painful irony is enough to give most system administrators a measure of respect for the bleeding edge of software.

When it comes to adopting new technology healthy skepticism is a good thing, but if overdone it can block the path of innovation and the payoff valuable technology. This is because in spite of the afore mentioned axiom, when it comes to reputable software ‘fresh is best’. If this were not true software versions would be valued for their age, like fine wine. Instead, for most commercial software, six months without an update is usually considered the early warning signs of a dying product.

The fear of early adoption is not limited to the implementation of entire systems, it also extends to installing new versions and using new features of existing systems. The gravity of ‘not broken / don’t fix it’ in a production environment sometimes results in shops running software that is several revisions behind and often out of production.

So how do you decide where to place yourself on the luddite / suicide scale of software deployment? The decision is different for each implementation, but we have some tools that can help you make your mind up. First, read the PaperCut Release History to find out about changes that have been made since the version that you are currently running. The list of new features, fixes and improvements are updated with each minor release. Second, when you find a feature that looks interesting (for example, maybe you like the idea of improving your printer utilization using some of the example recipes in PaperCut’s new advanced scripting), use the search tool on our website to search all of our documentation. You will find links to the Tour, Knowledge Base, and User Guide sections that were updated when the newly added features were introduced. Third, try it. The license allows you to install a copy of PaperCut on a test server (even a desktop system will do) so you can test features before introducing them into your production environment.

One last thing. If you are running an older version of PaperCut, please remember to check that your license will work with the new version. From the Admin Console open the About tab. You will find the Licensed Version, and if you purchased Upgrade Assurance a ‘Software updates available until‘ date. If not, upgrade costs are modest and listed here. You will also find a link to check for and order updates as well as a link to email technical support in case you have any questions about the update.

CC image courtesy of cogdogblog on flickr