Caching Hackage

| Comments

On several occasions I noticed that when performing a cabal update that the index was being downloaded at the rate of plus min 300 KB/s. Finally I got around to do something about this. I’ve set up a caching server located in Utrecht, The Netherlands. It is a caching proxy for the hackage repository. If you want to use it, add the following to your ~/.cabal/config file. (Or equivalent on Windows.)

1
remote-repo: cache:http://spockz.nl:12080/packages/archive

Be sure to comment out the already existing remote-repo. Otherwise, cabal will download both indexes and merge them, and we don’t want this.

The funny bit

Apparently this only helps if your machine is fast enough to process the index (untarring and all extra administration cabal performs).

Plainly getting the file from the cache:

1
2
3
4
> wget http://spockz.nl:12080/packages/archive/00-index.tar.gz
Saving to: 00-index.tar.gz
100%[======================================>] 3.394.821   10,9M/s   in 0,3s
2011-11-13 00:12:35 (10,9 MB/s) - 00-index.tar.gz saved [3394821/3394821]

And running cabal update with my cache as source:

1
cabal update -v3  7,50s user 0,21s system 99% cpu 7,736 total

And then finally, with the original repository:

1
cabal update -v3  7,57s user 0,25s system 28% cpu 27,372 total

So here we see that the user time is roughly the same but you spent almost three times more seconds waiting for your coffee to get cold. Any further speed improvements for cabal update will probably require optimalisation of the code.

The caching server

I’m using Varnish to cache the request to hackage. And here is my config file. Please shoot if you see any improvements.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
# This is a basic VCL configuration file for varnish.  See the vcl(7)
# man page for details on VCL syntax and semantics.

backend backend_0 {
	.host = "hackage.haskell.org";
	.port = "80";
	.connect_timeout = 0.4s;
	.first_byte_timeout = 300s;
	.between_bytes_timeout = 60s;
}


acl purge {
  "localhost";
	"hackage.haskell.org";
}

sub vcl_recv {
    set req.grace = 120s;
    set req.backend = backend_0;

		set req.http.host = "hackage.haskell.org";

    if (req.request == "PURGE") {
        if (!client.ip ~ purge) {
            error 405 "Not allowed.";
        }
        return(lookup);
    }

    if (req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "DELETE") {
        /* Non-RFC2616 or CONNECT which is weird. */
        return(pipe);
    }

    if (req.request != "GET" && req.request != "HEAD") {
        /* We only deal with GET and HEAD by default */
        return(pass);
    }

    if (req.http.If-None-Match) {
        return(pass);
    }

    if (req.url ~ "createObject") {
        return(pass);
    }

    remove req.http.Accept-Encoding;

    return(lookup);
}

sub vcl_pipe {
    # This is not necessary if you do not do any request rewriting.
    set req.http.connection = "close";

}

sub vcl_hit {

    if (req.request == "PURGE") {
        purge_url(req.url);
        error 200 "Purged";
    }

    if (!obj.cacheable) {
        return(pass);
    }
}

sub vcl_miss {

    if (req.request == "PURGE") {
        error 404 "Not in cache";
    }

}

sub vcl_fetch {
    set beresp.grace = 120s;

    if (!beresp.cacheable) {
        return(pass);
    }
    if (beresp.http.Set-Cookie) {
        return(pass);
    }
    if (beresp.http.Cache-Control ~ "(private|no-cache|no-store)") {
        return(pass);
    }
    if (beresp.http.Authorization && !beresp.http.Cache-Control ~ "public") {
        return(pass);
    }

}

sub vcl_deliver {

}

lhs2TeX-hl Version 1.4.5 Has Been Released!

| Comments

I’ve just released a new version of lhs2TeX-hl: 1.4.5 and it includes some new features:

  1. Added support for recursively traversing includes of .lhs files
  2. The program now doesn’t fail completely when haskell-src-exts fails to parse a file. An error is reported and the program continues. :) A fmt file is still generated.
  3. functions that aren’t functions but constants are now given the tag `constant’
  4. Removed a faulty command from the list.
  5. Cleaned up some code. (Probably introduced other ugly code)
  6. Binary operators are now typeset better. (I hope :))

Please let me know what you think.

Installing Vacuum-cairo With the Help of HomeBrew

| Comments

I tried installing vacuum-cairo on OS X 10.6 but it failed. With the help of HomeBrew I installed the dependencies by doing the following:

Be sure to run brew update first.

Then install the dependencies:

1
2
3
4
5
6
brew install pkg-config
brew install glib
brew install librsvg
chmod u+w /usr/local/Cellar/gdk-pixbuf/2.23.5/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
brew install librsvg
brew install pango

Note the installing of librsvg twice, this is in accordance with https://github.com/mxcl/homebrew/issues/4970. If you don’t get an error after running the first install of librsvg you might have a new update brew of librsvg.

And finally install vacuum-cairo:

1
cabal install vacuum-cairo

Applescript Script to Emulate the Behaviour of the `start’ Command in the Windows CMD Prompt

| Comments

People have asked me how to launch a program through the terminal in the background but with it’s own terminal screen similar to the way the “start” command in the Windows CMD prompt works. Well here it is. Add the following two files (start.applescript) and start to your path, and make them executable (chmod +x {start.applescript,start}).

One point of notice is to always encapsulate strings with spaces that should be one parameter in quotes.

Haskell Code Completion for TextMate

| Comments

Today I released my script for basic code completion support in TextMate.

The package can be found on hackage and on github.

Please read the README for installation instructions. Currently only identifiers from interpreted source code is available. Future releases will aim at providing better code completion support by using either Scion or HaskellSrcExts, or both.

Generic Text Markup With CSS

| Comments

Most of you are familiar with CSS reset scripts to reset CSS behaviour to something uniform across several browsers. I have been searching for a good CSS stylesheet that would do the same for the formatting of my text. That is, to give the text on a site a markup, suitable for reading. As a great fan of LaTeX this markup should follow the default markup of LaTeX as closely as possible.

As I could not find such a stylesheet I build my own. It probably is far from finished, although I tried to cover the most common cases in this script.

In time there will be a git repository that will contain test cases and updated versions of this stylesheet.

I tested this script in the presence of the Meyer CSS reset, but it should also work without any CSS Reset present, or together with other CSS reset stylesheets.

Please let me know (either by mail or a reply) if you have any remarks!

UUAG TextMate Bundle Released

| Comments

I’m proud to announce the release of the TextMate bundle for UUAG. It is currently very simple and is based on the Haskell TextMate bundle by Jamis Buck.

The only quirk is that you need to add --| after every sem block, otherwise TextMate will consider everything afterward Haskell code. This token is used to indicate the end of the Haskell code that officially starts after the =. This has the effect that every | before every Constructor is coloured blue. If you want to avoid this, insert --| after every definition.

1
2
3
4
sem Foo
  | Bar lhs.cafe = ...
  | Hotel lhs.desk = ...
--|

Happy coding! (GitHub repository, develop: UUAG.tmbundle.tar.gz)

A Note on the Versionless HTML

| Comments

Yesterday WhatWG announced to drop the versioning of HTML. HTML is to be a living standard. This comes at a very inconvenient time I fear.

Over the past few years web standards have, albeit slowly, developed into some mature state: Most browsers support the current (x)HTML, CSS and JS standards to the letter and can do more. However, due to the fact that these standards have evolved so slowly new developments are not included in these standards. HTML5 (W3C and WhatWG) solves a lot of these problems, including features like <canvas> and <video>.

With WhatWG dropping the version number of HTML I foresee a descent back to the Dark Ages where every browser supported a different standard and supported the same standards differently, one browser would support feature X and not Y while the other supports Y but not X. This is a nightmare for developers: not being able to rely on the uniformity of the browsers of your visitors. This created necessities as writing specific CSS and JS code for specific browsers to `fix’ certain behaviour. These fixes mostly had to be done for Internet Explorer as it had at the time by far the largest install base.

Now that we have finally arrived at an era where all browsers support the same set of rather advanced techniques we decide to drop versions. In some cases it is not a disaster to not have a version number. Look at CSS, the changes that have been made to the standard were mostly additions. New operators and new properties and values have been added. These changes required a better parser, and the recognition of the new fields, but never did a semantic change. This, together with the tradition that CSS engines ignore the statements they don’t understand has led to developers being able to include new techniques in CSS files and thus implicitly setting the version of the CSS in the file. As nothing changed older and newer browsers behaved the same at the old statements and only the newer browsers acted on the new statements. So up to now the absence of explicit CSS versioning didn’t cause any problems for CSS.

However, with the absence of versions in HTML5 we have a problem. Not only does the HTML5 standard describe the structure of HTML but it also describes the JS that the browser should support. These standards are in contrast to CSS subject to the changing of their semantics. For example, a JavaScript function can get a different interface: different parameters and/or return values, but also its semantic proper might change.

This changing of semantics without version numbers causes a new nightmare for developers, as they have no way to be sure that their site will look the same across different browsers as they all may support a slightly different incarnation of the standard and on top of that the developer cannot be sure his site will look the same in the same browser a month from now, or even the next day, as the browser can switch to support a newer instance of the standard. As different versions of the same browser may have different engines that support different incarnations of the standard you now also have to worry about the differences between browser versions that your visitors use.

One might notice the presence of a lot of conditions in the above text and think “What is the fuss, it might all not happen.”. The main argument I would like to make here is this: a `living’ standard introduces a lot of uncertainties. At a time where certainties are just starting to come back, giving developers peace of mind, going back to living with uncertainties is certainly a descent into Dark Ages in my opinion.

A solution?

There is of course a reason why they choose to declare this standard as `living’. It denotes the fact that the web is evolving on a faster pace than we are used to, it is quite dynamic. Having to wait several years before you can use a very nice new feature on your website is not nice, as you would like to use it as soon as it becomes defined. Therefore, instead of declaring the standard completely dynamic and deprived of versions, I suggest something like annual milestones. This way one can still compare browsers on their capabilities, new features can be expected to be available rather quickly and most importantly you provide a frame of reference for the developer.

lhs2TeX-hl 1.2.2 Released

| Comments

Changelog:

  1. Classes are written to the format file;
  2. There should be less “ambiguous operator” errors now;
  3. Removed the Agda dependency as it didn’t do anything with Agda at all.

Hackage, GitHub.

lhs2TeX-hl 0.1.1.1 Released

| Comments

After the first release of my tool on hackage this release actually is a working package. The previous one didn’t install out of the box.

The most important changes are:

  1. Added a filter for rewrite rules that aren’t “lhs2TeX safe”, e.g. format () = ...;
  2. I hardcoded the default formatting rules for several of lhs2TeX defaults, including but not limited to ->, <- and =>. This is not a desirable solution but it suits my purposes.

You can do literal (numeral, character and string) formatting easily with lhs2TeX itself with the following directives:

1
2
3
%subst char a    	= "\color{char}\text{\tt ''" a "''}"
%subst string a  	= "\color{string}\text{\tt \char34 " a "\char34}"
%subst numeral a =  "\color{numeral}{ " a " }"