Memcached + PHP
"i nie tylko"
(by Rafał 'after.design' Malinowski)
Kim Jestem ?
Rafał „after.design” Malinowski
- Programista JavaScript/PHP/Python
- Scrum master
- Project manager
- Product manager
"A kogo potrzeba ? ;)"
Plan
- memcached + php
- "i nie tylko"
Memcached - kto z tego korzysta ?
Memcached - co to jest ?
Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
Memcached is an in-m ory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.
Memcached - po ludzku fajny
- to jest "tylko" serwer
- key/value storage
- instancje nie są świadome swojego istnienia
- prosty w użyciu protokół (binarny i tekstowy)
Memcached - po ludzku nie fajny
- brak autoryzacji/ssl
- brak ustandarozywanych algorytmów
- brak ustandaryzowanych bibliotek
- klucze max 250 znaków
Metody memcached
- set
- add
- replace
- append
- prepend
- cas
- get
- gets
- delete
- incr/decr
- stats
- stats items
- stats slabs
- stats sizes
- flush_all
Protokół binarny
Byte/ 0 | 1 | 2 | 3 |
/ | | | |
|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
+---------------+---------------+---------------+---------------+
0| Magic | Opcode | Key length |
+---------------+---------------+---------------+---------------+
4| Extras length | Data type | vbucket id |
+---------------+---------------+---------------+---------------+
8| Total body length |
+---------------+---------------+---------------+---------------+
12| Opaque |
+---------------+---------------+---------------+---------------+
16| CAS |
| |
+---------------+---------------+---------------+---------------+
Total 24 bytes
Zemsta programisty
/* use lowest byte for flags */
#define MMC_SERIALIZED 0x0001
#define MMC_COMPRESSED 0x0002
/* use second lowest byte to indicate data type */
#define MMC_TYPE_STRING 0x0000
#define MMC_TYPE_BOOL 0x0100
/*#define MMC_TYPE_INT 0x0200*/
#define MMC_TYPE_LONG 0x0300
/*#define MMC_TYPE_DATE 0x0400*/
/*#define MMC_TYPE_BYTE 0x0500*/
/*#define MMC_TYPE_FLOAT 0x0600*/
#define MMC_TYPE_DOUBLE 0x0700
/*#define MMC_TYPE_BLOB 0x0800*/
Algorytmy wyboru serwera #1
modulo
- crc32($key) % count($servers)
- fnv-1a($key) % count($servers)
- migracja danych
- kolizje (fnv-1a)
Algorytmy wyboru serwera #2
Consistent Hashing (php-memcache)
$serverData = array("10.0.1.6:11211",
"10.0.1.7:11211");
$consistentPoints = 160; (php-memcache)
$serverHash = crc32('10.0.1.6:11211'.'-');
$server = '10.0.1.6:11211';
$points = array();
$points[crc32(0, $serverHash)] = $server;
.
.
.
$points[crc32(n, $serverHash)] = $server;
$points = array();
$points[262294557] = '10.0.1.6:11211';
$points[2024111243] = '10.0.1.6:11211';
$points[3786157361] = '10.0.1.6:11211';
$points[416897118] = '10.0.1.7:11211';
$points[1876846792] = '10.0.1.7:11211';
$points[4141300082] = '10.0.1.7:11211';
$sortedPoints = array_keys($points);
sort($sortedPoints);
$sortedPoints = array();
$sortedPoints[0] = 262294557;
$sortedPoints[1] = 416897118;
$sortedPoints[2] = 1876846792;
$sortedPoints[3] = 2024111243;
$sortedPoints[4] = 3786157361;
$sortedPoints[5] = 4141300082;
$consistentBuckets = 1024; (php-memcache)
$pointSearch = 0xffffffff / 1024;
//0xffffffff -> (pow(2,32)-1, max int)
$buckets[0] = search($pointSearch * 0);
.
.
.
$buckets[n] = search($pointSearch * n);
search(4194303 * 0);
$sortedPoints[0] = 262294557;
$sortedPoints[0] > 0; //True
$buckets[0] = $points[$sortedPoints[0]];
$buckets[0] = '10.0.1.6:11211';
search(4194303 * 100);
$sortedPoints[1] = 416897118;
$sortedPoints[2] = 1876846792;
$sortedPoints[1] > 419430300; //False
$sortedPoints[2] > 419430300; //True
$buckets[100] = $points[$sortedPoints[2]];
$buckets[100] = '10.0.1.6:11211';
$key = md5(1);
$key = "c4ca4238a0b923820dcc509a6f75849b";
$searchKey = crc32($key);
$server = $buckets[$searchKey % 1024];
//coś w stylu "10.0.1.6:11211"
- mniejsza migracja danych
Biblioteki
- php-memcache 3.0.6 beta w repo debiana
- php-memcached 1.0.2 stable w repo debiana
- php-memcached 2.0.0 beta - github
memcache | memcached | |
append/prepend | x | |
set/get per serwer | x | |
incr/decr | x | x |
dane liczbowe | konwersja | x |
- brak zgodności algorytmów wyboru serwera
- brak zgodności kompresowania danych
- zgodność typów danych ?
Memcache i zend framework
resources.cachemanager.cache.frontend.name = Core
resources.cachemanager.cache.frontend.options.lifetime = 1800
resources.cachemanager.cache.frontend.options.automatic_serialization = true
resources.cachemanager.cache.backend.name = Memcached
resources.cachemanager.cache.backend.options.servers.1.host = 10.20.0.122
resources.cachemanager.cache.backend.options.servers.1.tcp_port = 11211
resources.cachemanager.cache.backend.options.servers.1.udp_port = 0
resources.cachemanager.cache.backend.options.servers.1.persistent = true
resources.cachemanager.cache.backend.options.servers.1.weight = 1
resources.cachemanager.cache.backend.options.servers.1.timeout = 1
resources.cachemanager.cache.backend.options.servers.1.retry_interval = 15
resources.cachemanager.cache.backend.options.servers.1.status = true
Czy potrzebóję memcached ?
Nie ?
"premature optimization is the root of all evil" (Donald Knuth)
Wyjątek ?
Shardowanie baz danych
Najlepsza biblioteka ?
Nie istnieje ?
Co jest potrzebne ?
Coś oparte o libmemcached
A i tak nie będzie zgodne między językami.