Daniel Brunner: Mein Blog: Posts tagged 'Programmierung'urn:https-www-dbrunner-de:-tags-Programmierung-html2020-12-06T15:52:51ZDie Programmiersprache F#urn:https-www-dbrunner-de:-blog-2020-12-06-die-programmiersprache-f2020-12-06T15:52:51Z2020-12-06T15:52:51ZDaniel Brunner
<p>Im Rahmen einer Studienleistung für eine Fort- und Weiterbildung habe ich an der FernUniversität in Hagen eine Ausarbeitung zu der Programmiersprache F# angefertigt. Vielleicht hilft die Darstellung der einen oder anderen, sich einen kurzen Überblick zu verschaffen. Zu finden ist sie <a href="/pub/2020-09-30-fsharp.pdf">hier.</a></p>Evaluierungsergebnisse mit Racket/Scribble verarbeiten und darstellenurn:https-www-dbrunner-de:-blog-2017-12-11-evaluierungsergebnisse-mit-racket-scribble-verarbeiten-und-darstellen2017-12-11T16:58:35Z2017-12-11T16:58:35ZDaniel Brunner
<p>Auch in diesem Jahr habe ich (zum Teil mit Stephan Brunner zusammen) wieder Lehrveranstaltungen für StudiumPlus durchgeführt.</p>
<p>Die Ergebnisse finden sich jeweils als Link auf der <a href="/ueber/lehre.html">Seite mit meinen Lehrveranstaltungen</a>. Im Folgenden beschreibe ich, wie ich in diesem Jahr die Ergebnisse der Evaluierung mit Racket und Scribble verarbeitet habe.</p>
<!-- more-->
<p>Im vergangenen Jahr wurde eine zentrale Evaluierung der Lehrveranstaltungen von StudiumPlus durchgeführt. Die Studierenden konnten zu 13 Fragen ihre Einschätzung von “trifft voll zu” über “trifft meistens zu”, “trifft teilweise zu”, “trifft selten zu” zu “trifft überhaupt nicht zu” abgeben. Von der Hochschule erhielten wir dann eine Auswertung mit der Anzahl der jeweils gegebenen Einschätzungen pro Frage sowie der relativen Häufigkeit.</p>
<p>Nachdem ich in diesem Jahr eine Übung im Rahmen einer Vorlesung übernommen hatte, wusste ich nicht, ob dieser Teil eine eigene Evaluierung erhalten würde. Des Weiteren wurde für die Zwei-Tages-Module in diesem Jahr keine zentrale Evaluierung angeboten.</p>
<p>Ich habe daher die Fragen vom vergangenen Jahr mit dem Projektor an die Wand projeziert und die Studierenden um ihre Einschätzung gebeten.</p>
<p>Da ich die Ergebnisse verschiedentlich weiterverwenden wollte, habe ich mich entschieden, diese in Racket zu verarbeiten und mit Scribble in ein Dokument (sowie auch Seiten hier auf der Homepage) zu setzen. Der Code hierfür ist als Racket-Datei <a href="/lehre/format-eval.rkt">hier</a> zu finden.</p>
<p>Die Antworten führe ich dann als Liste von Arrays (Vector in Racket).</p>
<div class="brush: racket">
<pre><code> (list (vector 3 4 3 1 2 1 2 1 2 3 1 1 2) ; 1
(vector 2 1 1 1 2 1 1 1 2 #f 1 1 1) ; 2
(vector 1 4 1 1 2 1 2 2 2 3 1 1 1) ; 3
(vector 2 3 1 1 2 2 2 2 2 2 1 1 2) ; 4
...)</code></pre></div>
<p>Um die Häufigkeiten für eine Frage zu ermitteln, extrahiere ich mittels <code>vector-ref</code> und <code>map</code> sämtliche Antworten zu einer Frage. Hat ein Studierender nicht oder nicht leserlich geantwortet, habe ich jeweils <code>#f</code> eingetragen, die ich aus der Ergebnisliste entferne:</p>
<div class="brush: racket">
<pre><code>(define (sample-question loe no)
;; Gibt alle Antworten auf eine Fragenummer
(remove* (list #f) (map (lambda (answers) (vector-ref answers no)) loe)))</code></pre></div>
<p>Als Ergebnis erhalte ich eine Liste mit Antworten (wobei die Antworten derart kodiert sind, dass eine <code>1</code> für “trifft voll zu” und eine <code>5</code> für “trifft überhaupt nicht zu” steht), zum Beispiel:</p>
<div class="brush: racket">
<pre><code>'(4 1 4 3 3 5 4 3 3 4 3 2 4 1 4 1 3 3 2 2 2)</code></pre></div>
<p>Für die Weiterverarbeitung habe ich im <code>math</code> Modul die praktische Funktion <a href="http://docs.racket-lang.org/math/stats.html#%28def._%28%28lib._math%2Fstatistics..rkt%29._samples-~3ehash%29%29"><code>samples->hash</code></a> gefunden, die einem das Zählen abnimmt und einen Hash-Table mit der absoluten Häufigkeit zurückgibt:</p>
<div class="brush: racket">
<pre><code>> (samples->hash '(4 1 4 3 3 5 4 3 3 4 3 2 4 1 4 1 3 3 2 2 2))
'#hash((4 . 6) (3 . 7) (2 . 4) (1 . 3) (5 . 1))</code></pre></div>
<p>Hieraus habe ich dann eine Formatierung in eine Liste zusammengestellt, die anschließend in ein <code>tabular</code> eingebettet werden kann:</p>
<div class="brush: racket">
<pre><code>> (result EVALUIERUNG 0)
'(("Bewertung" "++" "+" "0" "-" "- -")
("Anzahl" "5" "7" "8" "2" "0")
("in v.H." "22.73" "31.82" "36.36" "9.09" "0.00"))</code></pre></div>
<p>Etwas gekämpft hatte ich einige Zeit mit dem Umstand, dass ein sogenannter <code>content</code> ein String sein muss und nicht einfach eine Zahl. Man muss also die vorher ausgerechneten Werte via <code>number->string</code> in einen String umwandeln, damit Scribble diese verwenden kann.</p>
<p>Da ich die Parameter für die Tabular-Umgebung auch nicht immer wieder tippen wollte, habe ich hierfür auch eine Funktion geschrieben, so dass es im Scribble-Dokument wie folgt aussieht (gegeben, dass sich hinter EVALUIERUNG die Liste der Arrays mit den Antworten verbirgt):</p>
<div class="brush: scribble">
<pre><code>@itemlist[#:style 'ordered @item{Die Veranstaltung ist klar strukturiert.
@tabular-result[EVALUIERUNG 0]
}
@item{Die verwendeten Veranstaltungsunterlagen sind hilfreich.
@tabular-result[EVALUIERUNG 1]}
...</code></pre></div>
<p>Das Ergebnis kann man am Beispiel der letzten Veranstaltung <a href="/lehre/iot-eval-2017.html">hier</a> einsehen, die zugehörige Scribble-Datei ist <a href="/lehre/iot-eval-2017.scrbl">hier</a> einsehbar.</p>
<p>Der Vorteil für mich liegt bei dieser Vorgehensweise darin, dass ich die Antworten in strukturierter Form behalte und auch noch für andere Zwecke weiter verwenden kann. Darüberhinaus ist diese Ausgabe mittels Scribble vielfältig nutzbar: Ich erzeuge bspw. daraus ein PDF-Dokument (über LaTeX) und eine Seite hier auf der Homepage mit dem statischen Seitengenerator Frog. Und, ich musste keine Tabellenkalkulation verwenden.</p>Using Racket Minimal and racourn:https-www-dbrunner-de:-blog-2016-01-12-using-racket-minimal-and-raco2016-01-12T09:55:43Z2016-01-12T09:55:43ZDaniel Brunner
<p>I use Racket Minimal on my smart phone (<a href="../2015/08/27/how-to-run-racket-on-the-raspberry-pi-2/">this</a> describes how to compile the run time for an ARM based system). It’s is a very small installation of Racket (about 36 MB after installation). After installation one only needs to install the packages that are really neded. But this can be a bit tricky because a lot of packages want to install their documentation and other stuff and bring a whole bunch of files on your drive as dependencies.</p>
<p>Some of the packages are divided up into a "-lib", "-doc" (and sometimes "-test") as laid out in the <a href="https://docs.racket-lang.org/pkg/getting-started.html#%28part._.Naming_and_.Designing_.Packages%29">documentation</a>. With these packages it’s easier to only install the implementation.</p>
<p>A small script of mine used only basic modules and relied on <code>rackunit</code> for the tests. On a mobile device the start up time of such a program can be critical. Therefore it is wise to only require the needed packages and to have the source code being compiled to byte code. One could do this with <code>raco setup</code> (which is included in Minimal Racket) but I wanted to have <code>raco make</code> (which is not part of Minimal Racket) available.</p>
<p>The commands of <code>raco</code> are added via a <code>raco-commands</code> variable in packages’ <code>info.rkt</code> file. I looked through the packages of my “full install” and found the package <code>compiler-lib</code> which adds some commands (<code>make</code>, <code>exe</code>, <code>pack</code>, <code>unpack</code>, <code>decompile</code>, <code>test</code>, <code>expand</code>, <code>read</code>, <code>distribute</code>, <code>demodularize</code>) to <a href="https://mirror.racket-lang.org/releases/6.3/doc/raco/index.html"><code>raco</code></a> and relies on only a few other packages. As a result the source and binary files need about 3.8 MB on my phone which is okay for me.</p>
<p>To sum up: After a simple <code>raco pkg install compiler-lib</code> I could easily use <code>raco make</code> and <code>raco test</code> to play with my program on my phone.</p>I played with CHICKEN Scheme, Docker and Alpine Linuxurn:https-www-dbrunner-de:-blog-2015-12-19-i-played-with-chicken-scheme-docker-and-alpine-linux2015-12-19T16:54:50Z2015-12-19T16:54:50ZDaniel Brunner
<p>I am looking forward to meet LISP people at the <a href="https://events.ccc.de/congress/2015/wiki/Main_Page">32c3’s</a> <a href="https://events.ccc.de/congress/2015/wiki/Assembly:The_%28un%29employed_schemers_%26_lispers_guild">LISP assembly</a>. The last days I played a bit with different Scheme implementations including <a href="http://call-cc.org">CHICKEN scheme</a>. The main feature of CHICKEN is that it compiles the Scheme code to C and then creates dynamic libraries and binaries with the C compiler. I thought that combining these binaries with a minimal Docker container could give me a very small deployment. So here are my steps:</p>
<!-- more-->
<h2 id="choosing-alpine-linux-as-a-small-linux">Choosing Alpine Linux as a “small” Linux</h2>
<p>The smallest Linux image for Docker is undoubtly busybox with a size of about 2.489 MB. But busybox lacks a package manager which makes installing software painful. Therefore I have chosen <a href="http://alpinelinux.org">Alpine Linux</a> which comes with package manager and it’s image’s size is about 5.234 MB. That’s double the size of the busybox image but still quite small compared to the Ubuntu image which is about 266 MB.</p>
<h2 id="creating-a-docker-container-with-chicken">Creating a Docker container with CHICKEN</h2>
<p>Alpine Linux comes with the <a href="http://www.muscl-libc.org">musl libc</a> and I thought it would be best to compile all the CHICKEN stuff with that libc. Therefore I created a Docker container with gcc and all the other stuff with this Dockerfile (<a href="https://github.com/krrrcks/chicken-docker-alpine">Github repository</a>):</p>
<pre><code>FROM alpine:3.2
RUN apk update && apk add make gcc musl-dev
RUN wget -O - http://code.call-cc.org/releases/4.10.0/chicken-4.10.0.tar.gz | tar xz
WORKDIR /chicken-4.10.0
RUN make PLATFORM=linux && make PLATFORM=linux install
RUN rm -fr /chicken-4.10.0
WORKDIR /
CMD ["csi"]</code></pre>
<p>This image is quite big (about 161.7 MB) and is available for download at the <a href="https://hub.docker.com/r/krrrcks/chicken-alpine/">Docker Hub</a>.</p>
<h2 id="compiling-some-chicken-code">Compiling some CHICKEN code</h2>
<p>For testing purposes I wanted a minimal web server running in the Alpine Linux image. Therefore I looked through the <a href="http://wiki.call-cc.org/chicken-projects/egg-index-4.html">egg index</a> and found <a href="http://wiki.call-cc.org/eggref/4/spiffy">spiffy</a>. I fired up the <code>chicken-alpine</code> container (but I used <code>ash</code> as command instead of the <code>csi</code> Scheme interpreter) and created a small web server that serves some static pages. I wrote a <code>main.scm</code>:</p>
<pre><code>(use spiffy)
(start-server)</code></pre>
<p>and added some static pages to a <code>./web</code> sub-directory. Then everything had to be compiled and prepared for deployment:</p>
<pre><code>chicken-install spiffy
csc -deploy main.scm
chicken-install -deploy -p $PWD/main spiffy</code></pre>
<h2 id="deploy-in-a-fresh-alpine-linux-image">Deploy in a fresh Alpine Linux image</h2>
<p>After the compilation I copied the <code>main</code> and <code>web</code> directories on my host machine using <code>docker cp</code> and created the following Dockerfile:</p>
<pre><code>FROM alpine:3.2
ADD main /main
ADD web main/web
WORKDIR main
CMD /main/main</code></pre>
<p>and let <code>docker build -t krrrcks/spiffy-test .</code> do the job. The size of the resulting image is about 12.37 MB and that’s pretty small. I uploaded that image to the <a href="https://hub.docker.com/r/krrrcks/spiffy-test/">Docker Hub</a> as well.</p>
<p>To serve the pages I did a <code>docker run -d -p 8080:8080 krrrcks/spiffy-test</code> (spiffy listens on port 8080 in the default install) and browsed my static pages.</p>How to use GET Bucket location on Amazon S3 with Racketurn:https-www-dbrunner-de:-blog-2015-09-04-how-to-use-get-bucket-location-on-amazon-s3-with-racket2015-09-04T05:23:43Z2015-09-04T05:23:43ZDaniel Brunner
<p>In <a href="http://www.racket-lang.org">Racket</a> I want to iterate over my buckets in Amazon S3. They are located in different regions. So how do I get my bucket’s location/region? In the API Reference there is a call <a href="http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETlocation.html">GET Bucket location</a>. I use <a href="https://github.com/greghendershott/aws">Greg’s AWS library for Racket</a> and this library authenticates its calls with <a href="http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html">signature version V4</a>. But V4 requires the user to know the <em>region</em> to correctly sign the request. So I need to know the region to ask Amazon S3 for the region where the bucket is located. Otherwise Amazon S3 responds with an error:</p>
<pre><code><?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>AuthorizationHeaderMalformed</Code>
<Message>The authorization header is malformed; the region 'us-east-1'
is wrong; expecting 'eu-central-1'</Message>
<Region>eu-central-1</Region>
<RequestId>XXXX</RequestId>
<HostId>XXXX>
</Error></code></pre>
<p>After some search on the net I found a <a href="http://stackoverflow.com/questions/27091816/retrieve-buckets-objects-without-knowing-buckets-region-with-aws-s3-rest-api">post on Stackoverflow</a> that helped to solve that issue: If I use the URL format (instead of the normally used virtual host format) I could get the location of any bucket. Every region responds with a <em>LocationConstraint</em> answer.</p>
<p>Therefore a code snippet for Racket could be:</p>
<pre><code>(define (get-bucket-location bucket)
(parameterize
([s3-path-requests? #t])
(define xpr (get/proc (string-append bucket "/?location") read-entity/xexpr))
(and (list? xpr)
(= (length xpr) 3)
(third xpr))))</code></pre>
<p>For example:</p>
<pre><code>> (get-bucket-location "my-bucket-somewhere")
"eu-central-1"</code></pre>
<p>PS: I think official Amazon S3 documentation could be a bit more verbose on the issues with GetBucketLocation and signature V4.</p>
<p><em>Update:</em> Greg added a <code>bucket-location</code> function to his great <a href="http://docs.racket-lang.org/aws/S3__Storage_.html#%28def._%28%28lib._aws%2Fs3..rkt%29._bucket-location%29%29">library</a></p>Running Racket on AWS Lambdaurn:https-www-dbrunner-de:-blog-2015-08-27-running-racket-on-aws-lambda2015-08-27T10:46:57Z2015-08-27T10:46:57ZDaniel Brunner
<p>I started to use AWS for some projects recently. But I only use few of their services. From time to time I look into some of there services and wonder if they are useful for my tasks. I looked into <a href="http://aws.amazon.com/lambda">AWS Lambda</a>, "… a compute service that runs your code in response to events and automatically manages the compute resources for you, making it easy to build applications that respond quickly to new information." Nowadays these “lambda functions” could be written in NodeJS or Java. When I was looking for a roadmap of the supported languages I found an interesting <a href="http://blog.0x82.com/2014/11/24/aws-lambda-functions-in-go/">blog post</a> by <a href="https://www.twitter.com/rubenfonseca">Ruben Fonseca</a>. He explaind how to run Go code on AWS Lambda.</p>
<p>I tried the same with <a href="http://racket-lang.org">Racket</a> and wrote a short Racket programm <code>test.rkt</code>:</p>
<pre><code>#lang racket/base
(display (format "Hello from Racket, args: ~a~%" (current-command-line-arguments)))</code></pre>
<p>Then I used <code>raco</code> to create a binary <code>test</code>:</p>
<pre><code>raco exe --orig-exe test.rkt</code></pre>
<p>I took the NodeJS wrapper from Ruben’s blog post and put it in a file <code>main.js</code>:</p>
<pre><code>var child_process = require('child_process');
exports.handler = function(event, context) {
var proc = child_process.spawn('./test', [ JSON.stringify(event) ], { stdio: 'inherit' });
proc.on('close', function(code) {
if(code !== 0) {
return context.done(new Error("Process exited with non-zero status code"));
}
context.done(null);
});
}</code></pre>
<p>Then I put both files in a zip archive, created a new AWS Lambda function, uploaded the zip file and invoked the function:</p>
<div class="figure"><img src="/img/2015-08-27-racket-aws-lambda.png" alt="Invocation of AWS Lambda function" />
<p class="caption">Invocation of AWS Lambda function</p></div>
<p>Fine!</p>
<p>PS: Only question is: When is AWS Lambda coming to the region <code>eu-central-1</code>, located in Frankfurt?</p>
<p><em>Upate (2016–03–15):</em> AWS Lambda is <a href="https://aws.amazon.com/de/about-aws/whats-new/2016/03/aws-lambda-available-in-eu-frankfurt/">now available</a> in the EU (Frankfurt) region!</p>DateTime conversion can be trickyurn:https-www-dbrunner-de:-blog-2014-07-24-datetime-conversion-can-be-tricky2014-07-24T07:41:36Z2014-07-24T07:41:36ZDaniel Brunner
<p>I wrote a small Lisp application and a JavaScript client gets some data from that application. All time stamps are returned as “Lisp” time stamps, i.e. an integer with seconds where zero equals Jan 01 1900.</p>
<p>In the JS client the time stamp is then converted to JS time stamps, i.e. millisconds where zero equals Jan 01 1970.</p>
<p>When testing the application I noticed that sometimes the displayed date is one day behind. For example in the data base I have Jan 05 1980 but in JavaScript I get a Jan 04 1980. But some other dates worked: A time stamp Jan 05 1970 was correctly converted to Jan 05 1970.</p>
<p>I had a look into the JavaScript code and found:</p>
<pre><code>convA = function(ts) {
tmp = new Date(ts*1000);
tmp.setFullYear(tmp.getFullYear() - 70);
return tmp.getTime();
}</code></pre>
<p>It’s likely the developer thought: “Well, it’s millisecond instead of second. Therefore I multiply by 1,000. But then I am 70 years in the future and I have to substract 70 years and everything will be ok.”</p>
<p>After thinking a while I came to the conclusion: Of course not!</p>
<p>The developer made the assumption that there are as many leap years between 1900 and 1970 as between <code>ts</code> and <code>ts+70</code>. Obviously that assumption does not hold for all time stamps. And therefore sometimes the resulting JavaScript date is one day behind.</p>
<p>So a better solution would be to substract all seconds between 1900 and 1970 from <code>ts</code>, multiply by 1,000 and treat this as a JavaScript time stamp. Perhaps best would be to do the conversion in the Lisp process and only deliver a JavaScript-like time stamp.</p>I learned something about symbols and packagesurn:https-www-dbrunner-de:-blog-2014-07-06-i-learned-something-about-symbols-and-packages2014-07-06T07:02:39Z2014-07-06T07:02:39ZDaniel Brunner
<p>I am using Common Lisp for developing a web application. Several days ago a new part of this application didn’t worked as supposed and I spent a considerable large amount of time in finding the bug. It was a very simple problem with symbols where I mixed something up.</p>
<p>In the application the web server somewhen gets some JSON data from the browser. It is then converted to Lisp object using the <code>CL-JSON</code> package. This package converts JSON objects to a-lists and converts the member keys to symbols (see CL-JSON’s <a href="http://common-lisp.net/project/cl-json/">documentation</a>. I then wanted to look something up in that a-list and failed.</p>
<p>I wrote a small test case to show the effect and explain what went wrong.</p>
<pre><code>(ql:quickload '("hunchentoot" "cl-who"))
;; direct loading via ql only for demonstration purposes, normally I
;; would use a asdf:defsystem for that.
(in-package :cl-user)
(defpackage :my-app (:use :cl))
(in-package :my-app)
(defparameter *my-a-list*
'((foo . 100)
(bar . 200))) ;; in the real application this a-list is
;; generated by a JSON-to-lisp conversion by
;; CL-JSON; in CL-JSON the object member keys are
;; converted to symbols.
(defun get-value (key)
"Returns the value with KEY from *MY-A-LIST*."
(cdr (assoc (intern (string-upcase key)) *my-a-list*)))
(hunchentoot:define-easy-handler (web-get-value :uri "/get-value") (id)
(cl-who:with-html-output-to-string (*standard-output* nil :prologue t)
(:p (cl-who:fmt "Value of ~a is: ~a" id (get-value id)))))
(defun start ()
(hunchentoot:start (make-instance 'hunchentoot:easy-acceptor :port 4242)))</code></pre>
<p>So on the REPL everything looks fine: <code>MY-APP> (get-value "foo")
100
MY-APP> (get-value "bar")
200
MY-APP></code></p>
<p>But when I used my web browser to give me these results as well I got something strange. For example here are some results when using curl: <code>~> curl http://localhost:4242/get-value?id=foo
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<p>Value of foo is: NIL</p></code></p>
<p>I was puzzled: The value is <code>NIL</code>?</p>
<p>After some debugging I found out that the easy handler from Hunchentoot runs with <code>*package*</code> set to <code>COMMON-LISP-USER</code> (and not to <code>MY-APP</code> as I implicitly assumed). That means that <code>assoc</code> looked up <code>COMMON-LISP-USER::FOO</code> in the a-list where the keys are <code>MY-APP::FOO</code> and <code>MY-APP::BAR</code>. And this test fails. Therefore <code>NIL</code> is returned which is correct.</p>
<p>So I rewrote the <code>get-value</code> function to: <code>(defun get-value (key)
"Returns the value with KEY from *MY-A-LIST*."
(cdr (assoc (intern (string-upcase key)
(find-package :my-app)) *my-a-list*)))</code> Now the symbols are interned in the same package and everything went well: <code>~> curl http://localhost:4242/get-value?id=foo
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<p>Value of foo ist: 100</p>
~> curl http://localhost:4242/get-value?id=bar
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<p>Value of bar ist: 200</p></code></p>
<p>Therefore I was reminded to think about packages when interning symbols. A good guide to symbols and packages could be found in this document: <a href="http://www.flownet.com/gat/packages.pdf">The Complete Idiot’s Guide to Common Lisp Packages</a>.</p>