Sonntag, 18. September 2016

https: use node.js spdy to implement http2

Because http2 is the next generation protocoll in the web with all its advantages like performance, better informations etc, we want to implement a simple http2 webserver.

Most of the browser do actually support http2. FF, Safari IE 11 and Opera in the latest version does.

Because all browsers are making ssl / tsl to the encryption standard when using http2, you will need a certificate.

To make things simple we selfsign this certificate and import it to firefox.

Prerequisits

We need some things installed on the local machine.

a /etc/hostfile entry for "nodejs.local"
a ssl certificate for nodejs.local
node.js installed

Get the ssl cert and the server key

Create a projectfolder in your webroot.

Save that cert as server.crt and the server.key to your local project under <projectroot>/etc/certs/

server.crt

server.key

Alternativly you can sign your own cert with openssl

Now open in Firefox:
Settings->Advanced->Certificate and import it under "Cert Authority"

Create a simple webapp using http2

Install spdy and fs

npm install --save fs spdy

Spdy is needed to handly http2 requests and fs will handle our key and certfiles

Create the webserver by saving folling code as index.js in your projectroot

const spdy = require('spdy');
const fs = require('fs');


const options =
{
    key:fs.readFileSync('./etc/certs/server.key'),
    cert: fs.readFileSync('./etc/certs/server.crt')
}

spdy.createServer(options, (req, res)=>
{
    res.writeHead(200);
    res.end('Hello Client');

}).listen(8084);

Run your new http2 server

node index.js

This will bind a simple server on port 8084, so that you can surf http://nodejs.local:8084 and see "Hello Client"

If you open firebug, you will see, that your request will use http 2

This is AWESOME

openssl: create a self signed certificate for a local host entry

From time to time you will need to create a TSL / SSL certificate on your local machine. For instance, if you want you use http2. In most of the browsers http2 implements TSL / SSL. So you need a cert for your localhost.

You will learn in this tutorial

to create a secure and a insecure serverkey needed for a Certificate Request with openssl
to create a Certificate Request from a serverkey with openssl
to create a selfsigned certificate from a Certificate Request with openssl

Prerequisits

openssl

Create serverkeys

The first step in certificate creatation is to create serverkeys for a CR (Certificate Request) on your servermachine. That means, your will create a 2048 bit RSA server.key on your local servermachine and reques a cert for for this server on a CA (Certificate Authority) so that the CA can approve this request with a cert for this servermachine key.

Every cert is signed to only one serverkey, except when you are using wildcard certificates

Normally you will use a common CA like Verisign, Commode or Geotrust. In our case we want to create a selfsigned certificate. In that case we are our own CA.

Clearly, thats only usefull, if we want to develop locally or internaly.

So let's create the server.key's

openssl genrsa -des3 -out server.key 2048

At next you will be asked for a passphrase. Type in a secure password with min length 5 letters, a specialchar and 1 digit

That's it. You will now find a file "server.key" in the current directory

Create a server.key for passwordless encryption.
The server.key you have cfreated will ask everytime for a pasword, if you use it. To prevent that, we will now create a further serer,key, which doens't need a password, because it's based on the password encrypted key, we have created before.

openssl rsa -in server.key -out server.key.insecure

This will save a new key "server.key.insecure"

Let's rename the key
To get rid of the insecure word in a serverkey, we simply rename them. :)

mv server.key server.key.secure

mv server.key.insecure server.key

Now we have created our passwordless server,key.

Create a CR for your server.key

Now we have to create our Certificate Request (CR) for our server.key. This can also be done by openssl

openssl req -new -key server.key -out server.csr

At next you will be asked for the server.key passphrase.

In the next prompts you will be asked for Detailinformations of your company.

We want to create a cert for local development under "https://nodejs.local".

So your FQDN is nodejs.local

Don`t forget to add nodejs.local to your local /etc/hosts

After you typed in your password,openssl will save a file "server.csr" in your current directory.

Now, you can send this CertRequest to your CertAuthority or you can self sign the certificate

Selfsign your certificate

For local development we do not want to spend 100 Bugs a year, So lets selfsign the cert. But you will never use that for stage or live / prod system, where your customers are logging them in.

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

Great. Now we have a new certificate for our dev environment

You can use the generated "the server.crt" and "server.key" files in your application to en- and decrypt requests

It is a good idea to store the files in a central folder like /etc/ssl/certs and /etc/ssl/keys

At last you have to import this new cert in your browser.

Under Settings->Advanced->Certificates you can inport it under "Certificate Authorities"

Cheers !

Freitag, 16. September 2016

jenkins: install an ubuntu slave as a service.

Before we start to write any line of code, we should make sure, that out buildsystem is running and the CI process in our new project is automatted.

The CI pipline only is complete, when you deploy your project directly to your test , stage and liveservern.

One possibility is, to register your webserver as a slave on your buildserver.

In our case we are using jenkins as a master build server and a ubuntu webserver as a slave.

If everythings is correct, you will see yout slavenode on the jenkinsdashboard.
Our slave is called "docker_gitlab_webserver_1" our Jenkins server listens to "dockergitlab_jenkins_1" on port 8080

So let'shave a look on the configuration

Configure your new node

As you can see we have defined a rootdirectory on the slave "/var/lib/jenkins".
Create that folder a on the slave machine nd make it accessable for the master.

This configuration will use Java Webstart with Jnpl. This will do the trick on communication between slave and master.

After you save the configuration you will get a downloadlink for your "slave.jar" and a command, which you can execute on your slave

Execute on your slave

java -jar slave.jar -jnlpUrl http://dockergitlab_jenkins_1:8080/computer/dockergitlab_webserver_1/slave-agent.jnlp

This has to be executed in "/var/lib/jenkins"

Make sure you have java 7 installed

After that you can see your slave running "connected" on your jenkins master.

Great !

Install this startupscript as a service

To make that rebootsafe you have to create a service and add it to the ubuntu runlevels S and 2, 3 and 6

Create a file "/etc/init.d/jenkins-slave"

sudo chmod 755 /etc/init.d/jenkins-slave
sudo chown root:root /etc/init.d/jenkins-slave

sudo update-rc.d jenkins-slave defaults

Add it to Autostart

sudo ln -s /usr/lib/insserv/insserv /sbin/jenkins-slave
sudo insserv jenkins-slave

This will set correct accessrights to your service Make sure, you have replaced the parameters with your data.

Sonntag, 11. September 2016

unittesting: install and use phpunit from scratch.

If you start a new php project, you will find it hard to implement phpunit in your project and in your local environment, because after a while you simply forget how the setup is working.

So let's repeat the installation and configuration of composer and phpunit.

We want:

running composer
running phpunit
a sample project skelleton
a sample class
a simple test for a class

We need some things installed on the local machine. We are webprogrammers who love unix and linux. So we are using a ubuntu xenial 16.04

Install composer

With composer the hard task of outloading your sourcefiles is a little bit simpler. Because it makes some work for us, we really dont want to do, like setup autoloading manually. PHP isnt very comfortable here. So we have to use a tool.

Composer install

php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
php composer-setup.php
php -r "unlink('composer-setup.php');"
sudo mv composer.phar /usr/local/bin/
sed -i "$ aalias composer='php /usr/local/bin/composer.phar' " ~/.bashrc . ~/.profile
source ~/.bashrc

This will install composer and add it to your console, so that you can simply type "composer"

Install phpunit

To make sure, we have the lastest stable version installed we use composer to istall phpunit.

phpunit install

composer  global require "phpunit/phpunit"

add it to the console

sudo ln -s  ~/.composer/vendor/phpunit/phpunit/phpunit   /usr/bin/

test it

composer install

phpunit --version

Configure psr-4 autoloading

We want to use the latest standard to load our classes with namespacing and psr-4.

To do that, we have to tell composer to use psr-4

open composer.json and add

"autoload": {
    "psr-4": {"TestPhpProject\\": "src/"}
},

This will search for all Classes with Namespace "TestPhpProject" in a folder "src"

Now we have to run a scanning for all classes in class src initialy to create the psr-4 classmap

recreate psr-4 autoloading classmap

composer dump-autoload

Create a testproject

Now that we have composer ready to load our classes automaticly, we have to setup our corresponding projectstructure:

Create folderstructure like that
TestPhpProject
|_src
|_test

Create a phpunit test class under "test" named MathTest.php with this content:

<?php

use TestPhpProject\Math;

class MathTest extends \PHPUnit\Framework\TestCase
{
    // test the talk method
    public function testAdd() {
        // make an instance of the user
        $math = new Math();

        // use assertEquals to ensure the greeting is what you
        $expected = 1;
        $actual = $math->add(1,2);
        $this->assertEquals($expected, $actual);
    }
}

We always create our test before we implement the class.

This simple testcase uses the TestPhpProject\Math class , we created before.

The class MathTest extends the UnitTest Framework class "TestCase" and implements a method testAdd(). This method refences a Mathobject and comares trhe result from the method add() with an expected result 1. This will fail.

If your first test fails, ist greate, because it must fail for the first time. All things fail first!

Learn to accept! It makes things very easy.

Create a simple class under "src" named Math.php with this content:

<?php

namespace TestPhpProject;

Class Math
{
    public function __construct()
    {
        $this->_test = '1';
    }

    public function add($x, $y)
    {
        return $x+$y;
    }
}

As you can see we have defined a namespace TestPhpProject, wich corresponds to our psr-4 autoloader we defined before. It has a constructor which sets the property test to 1 and a function add, which simply add 2 sumands and returns the result.

Pretty simple.

Tell phpunit to use our test directory and our composer autoloader to load our testsfiles automaticly.

Add a file phpunit.xml to your projectroot with this content

<?xml version="1.0" encoding="utf-8" ?><phpunit bootstrap="./vendor/autoload.php">

 <testsuites>

   <testsuite name="The project's test suite">

     <directory>./tests</directory>

   </testsuite>

 </testsuites>
</phpunit>

Run our first test

phpunit

Samstag, 10. September 2016

build: automate your php code reviews with ant tasks

Each php project needs a local build process, to check the code quality.
In PHP there are some standard tools to check.
- Codestyle
- Dependencies
- Complexity
- Syntaxerrors

and many more issues.

Its very handy to use tools like ant, to wrap up this QA tools. So you can automate the code review process and run this tool against your codebase.

Once you have ant installed, you can add a build.xml definition to your project. This definition orchestrates the single phptools to inspect your code and create docs. This tasks are called "Ant-Tasks"

After you have setup your build.xml you can simly start ant to run all tests.

Later you can add this ant tasks to your jenkins buildsystem to check every releaseversion for issues in quality.

Lets start.

Prerequisits

We need some things installed on the local machine.

ant
composer

Create a testproject

I have created a small testproject with just 1 class and 1 testclass in it.
But that's okay to demonstrate the case. If you take a look in the composer.json, you will see, that all buildtools will be installed during build. So you dont need to install any tool, except ant

Just clone the project:
git clone https://github.com/pboethig/testPHPProject.git

Create your build definition

We want to create a single task for each testing tool in our defintion.
Additionalyl we have to prepare the buildfolder-structure and setup the project with this build definition.

Her is mine. It lives directly in the project root.

As you can see, there is a task named "<target>" for each tool which gets run on the codebase in the src folder.

The single targets gets run by the 2 targets TEST and REPORT. Later your will start the REPORT TASK via ant in the console. This will automaticly start the TEST target, so that single targets can be executed.

My build.xml from the projectroot

Start your local build

After you have installed the tools and configured your build tasks to run that tools on your codebase

Open a terminal in your projectroot and run:

ant init

If you want to start a release build you simply can run:

After that you can see the results on the console and in the "build/logs" folder.

ant init package -DTAG_TO_BUILD=1.0.1

This will create a release packagein folder "release"

You can use this project-skelleton in jenkins buildjobs too.

For better understanding of the testresults it good to read the documentation of the tools.

Donnerstag, 8. September 2016

build: get a webapp buildenvironment up and running with one click. jenkis, selenium, gitlab, nexus out of the box

The foundation of a succesful product is a comlete automated infrastructure.
From editing a file till the automated deployment pipeline.

Only a nearly complete automated build-, test- , release- and deploymentstructure makes it possible to implement modern architectures, which saves time and let us concentrate on the important things to do, like writing businesscode

We want:

Versioncontroll with git
Codestylecheck with codesniffer
"Copy and Paste" detector to prevent mess
Automated unittest
Automatted seleniumtests
A releasepackage build
Copy the release artifacts to the artifact manager
autmate the deployment

For that we need some tools like gitlab, jenkins, sonar nexus and selenium. But noone want to install these tools by hand today.

"Git submodules" and "docker-compose" does the trick for us.

Let's create a tool, that will install these tools for us in a docker construct.

Prerequisits

We need some things installed on the local machine.

docker installed
docker-compose installed

Get the installscript

We just clone the docker scripts from this repository

Clone setupscripts

$ git clone --recursive https://github.com/pboethig/PhpBuildSystem.git

$ cd builsystem

$ ./startup.sh

This will clone all releated submodules in that main repository

Installed Components

The startup.sh script will execute docker-compose commands for each single application. This will bring up all the needed containers.

Use "docker ps" and "docker images" to show the images and containers.

Versioncontrol

GitLab Community Edition 8.11.4 b871b76
- http://localhost:10080
- username: admin@gitlabsample.com / gitlabadmin
- Github project: https://github.com/sameersbn/docker-gitlab

Buildsystem

Jenkins 2.0
- http://localhost:8081
- username: admin / admin
- Build your projects, run tests , package them and deploy your application

Artifactmanager

Sonar Nexus Repository Manager OSS 2.13.0-0:

http://localhost:8082
username: admin / admin
Its the artifactmanager to store releases and 3rdparty lib

Acceptenstest

Seleniumgrid
- http://localhost:4444/grid/console
- Its the artifactmanager to store releases and 3rdparty libs
- Git project: https://github.com/elgalu/docker-selenium

Mittwoch, 7. September 2016

jenkins: easy install jenkins 2.0 with docker

To make some exercises with docker, dockerfile and docker-compose, we want to automate the installation of jenkins 2.

The goal is to have a startup script which runs the composecommands and starts the jenkins server.

Prerequisits

We need some things installed on the local machine.

docker installed
docker-compose installed

Get the installscript

We just clone the docker scripts from this repository

Clone setupscripts

$ git clone https://github.com/pboethig/jenkins.git

$ cd jenkins

As you can see, there are 3 files in that folder. A docker-compose.yml with the minimum on imagedefinitions, a dockerfile, with the definition of the build, the network and the used ports.
And a small startupscript which runs the docker-compose commands.

Run the installer

Type in your terminal

$ ./startup.sh

That´s it.

Now you can reach your app under http://localhost:8081 on linux.

On Windows / Mac you can get the dockermachine ip with
http://<docker-machine-ip>:8081

Dieses Blog durchsuchen

Sonntag, 18. September 2016

Prerequisits

Get the ssl cert and the server key

Create a simple webapp using http2

Create serverkeys

Create a CR for your server.key

Selfsign your certificate

Freitag, 16. September 2016

Configure your new node

Great !

Sonntag, 11. September 2016

Install composer

Install phpunit

Configure psr-4 autoloading

Create a testproject

Samstag, 10. September 2016

Prerequisits

Create a testproject

Create your build definition

Start your local build

Donnerstag, 8. September 2016

Prerequisits

Get the installscript

Installed Components

Versioncontrol

Buildsystem

Artifactmanager

Acceptenstest

Mittwoch, 7. September 2016

Prerequisits

Get the installscript

Run the installer