Dancing In The Cloud

20 June 2017

Sam Batschelet

Developer, End Point Corporation

About Me

Sam Batschelet (hexfusion)
Developer, End Point Corporation
www.endpoint.com

Slides
https://github.com/hexfusion/talks

Agenda

gRPC

History

RPC

gRPC

Core Features

gRPC + protobufs

Under The Hood

gRPC services are defined by protobuf files.

syntax = "proto3";
package authpb;

// User is a single entry in the bucket authUsers
message User {
    bytes name = 1;
    bytes password = 2;
    repeated string roles = 3;
}

Build example

protoc --go_out=. *.proto

gRPC + protobufs

protoc --go_out=. *.proto

Google::ProtocolBuffers::Dynamic

See github.com/mbarbon/google-protobuf-dynamic

etcd

etcd best use

etcd v2

What they learned.

# create dir
etcdctl mkdir /backups

etcd v3

Features

Goals

Compare etcd Consol Zookeeper

Latency

Credit: github.com/gyuho/dbtester

CPU / Scaling

Credit: github.com/gyuho/dbtester

Throughput

Credit: github.com/gyuho/dbtester

Latency By Key

Credit: github.com/gyuho/dbtester

etcd v3

Issues

etcd v3

Solution grpc-gateway

Shout out: github.com/yugui

etcd v3 grpc-gateway

Credit: github.com/grpc-ecosystem/grpc-gateway

Experiment Net::Etcd

Net::Etcd

Net::Etcd

Feature walk through

Net::Etcd

KV Put/Range

# put
$etcd->put( { key => 'foo1', value => 'bar' } );

# range/get
$key = $etcd->range( { key => 'foo1' } )->key_value;

# delete
$etcd->range( { key => 'foo1' } )->delete;
Net::Etcd::KV

etcd Lease

A lease will expires if the server does not receive a keepalive within a given time to live (TTL) period.

etcd Lease

Use cases

Lease Examples

use Math::Int64;
$int64 = int64_rand();
$lease_id = int64_to_string($int64); # ie 7587821338341002662

# create/grant 20 second lease
$etcd->lease( { ID => $lease_id, TTL => 20 } )->grant;

# add lease to key
$etcd->put( { key => 'foo2', value => 'bar2', lease => $lease_id } )

# get ttl of lease returns -1 if expired
$ttl = $etcd->lease( { ID => $lease_id, keys => 1 } )->ttl

# reset lease ttl
$etcd->lease( { ID => $lease_id } )->keepalive

# revoke lease
$etcd->lease( { ID => $lease_id } )->revoke
Net::Etcd::Lease

Watch

Watch watches for events happening or that have happened.

Challenges

Watch Examples

# create watch for key foo using callback
$watch = $etcd->watch({ key => 'foo'}, sub {
    my ($result) =  @_;
    print STDERR Dumper($result);
})->create;

# put key
$etcd->put({ key => 'foo', value => 'bar' });
Net::Etcd::Watch

Watch

Test drive

...

Txn

Txn processes multiple requests in a single transaction.

Logical representation in Perl.

If ( $key eq 'foo' and $rev == 1 ) {
    success(\@$success_actions);
}
else {
    failure(\@$failure_actions);
}

Txn Example

# create put
$put = $etcd->put({ key => 'foo1', value => 'newbar', hold => 1 });

# create range delete
$range = $etcd->range({ key =>'foo2', hold => 1 })->delete

# define success action
push @success, $etcd->op({ request_put => $put });

# define failure action
push @failure, $etcd->op({ request_delete_range => $range });

# check revision number for key foo is 1
push @compare, $etcd->compare( { key => 'foo', target => 'CREATE', create_revision => '1' });

# bombs away!
$txn = $etcd->txn( { compare => \@compare, success => \@success, failure => \@failure } );
Net::Etcd::KV::Txn

Auth Issues

Auth

Not yet released 3.2.0+git

Download here v3.2.0_plus_git

Auth Example

# define user and password
$etcd = Net::Etcd->new( { name => 'root', password => 'toor' } );

# enable
$etcd->auth()->enable;
Net::Etcd::Auth

Users Roles (ACL)

Example:

# create new user
$etcd->user( { name => 'testing', password => 'baz' } )->add;

# create role
$etcd->role( { name => 'newrole' } )->add;

# give read permission to range
$etcd->role_perm(
  { name => 'newrole', key => 'foo1', range_end => 'foo99', permType =>'READ'} )->grant;

# attach role to user
$etcd->user_role( { user => 'testing', role => 'newrole' } )->grant;

Dancer

Dancer::Plugin::Etcd

Easy access to etcd REST API within Dancer.

Possible use cases

Dancer::Plugin::Etcd

Custom Build/Testing Application

Dancer::Plugin::Etcd

Dancer::Plugin::Etcd

Usage

get '/foo/alive' => sub {
    $tokens = shift;
    $etcd = etcd();
    $etcd->put({ key => 'alive', value => 'foo' });
    [ ... ]
};

Dancer::Plugin::Etcd

shepherd allows round trip storage of your YAML configs using a docker like naming system.

Usage

# put config and define version
shepherd put

# get latest config
shepherd get sam/ic6:latest

:Uses its own auth details in Dancer config.

Kubernetes

What is it?

Kubernetes

Credit: kubernetes.io/docs

Kubernetes

Install minikube

Dependencies

minikube start

# create 50G kvm instance with default network.
minikube start --disk-size 50g --vm-driver kvm --kvm-network default

minikube dashboard

Kubernetes Minikube

Docker

# reuse docker deamon
eval $(minikube docker-env)

# will show all docker containers within Kubernetes
docker ps

# build your docker file
docker build -t sam/myapp:v1 .

Net::Kubernetes

# connection
$kube = Net::Kubernetes->new(
    url=>'http://127.0.0.1:8080',
    username=>'foo',
    password=>'baz'
);

# list current pods
$pod_list = $kube->list_pods();

Net::Kubernetes

# dancer service
$kube->create_from_file('configs/dancer-service.yaml');

# dancer-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: dancer
  labels:
    run: dancer
spec:
  type: ClusterIP
  ports:
  - port: 3000
    protocol: TCP 
    name: plack
  externalIPs:
  - 123.123.123.123
  selector:
    run: dancer
---


# verify
$services = $ns->list_services;

Net::Kubernetes

# dancer service
$kube->create_from_file('configs/dancer-deployment.yaml');


# dancer-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: dancer
  labels:
    run: dancer
spec:
  replicas: 1
  template:
    metadata:
      labels:
        run: dancer
    spec:
      containers:
      - name: my-app
        image: sam/myapp:v1
        ports:
        - containerPort: 3000

Play Time!

Play Me

Thank you

Sam Batschelet

Developer, End Point Corporation

Use the left and right arrow keys or click the left and right edges of the page to navigate between slides.
(Press 'H' or navigate to hide this message.)