"What is Spark - Scala collections running on a cluster"
Martin Odersky
Saturday, April 23, 2016
Friday, April 22, 2016
CACHE - distributed cache
http://www.codeproject.com/Articles/21508/Distributed-Caching-Using-a-Hash-Algorithm
In computing, a distributed cache is an extension of the traditional concept of cache used in a single locale. A distributed cache may span multiple servers so that it can grow in size and in transactional capacity. It is mainly used to store application data residing in database and web session data.
Hashing algorithm that can be used in distributed caching of data in web farms or implementing a distributed hash table (DHT).
We have a layer of web servers that handle the requests and a pool of machines that are used for caching. Each web server should be able to access any of the cache machines. The tricky aspect here is that when we cache some data as a result of a request to a particular web server, we should be able to retrieve that data from any of the other web servers.
Here is what we can do. We want to cache key value pairs and then at a later stage get hold of a cached value by providing the corresponding key. The code below takes a key and produces a hash bit array using SHA1. Then after some transformations we derive an integer number. A given key always produces the same number. In addition, all numbers are uniformly distributed which allows us to "page" them and assign them to a given number of cache machines or "buckets".
In computing, a distributed cache is an extension of the traditional concept of cache used in a single locale. A distributed cache may span multiple servers so that it can grow in size and in transactional capacity. It is mainly used to store application data residing in database and web session data.
Hashing algorithm that can be used in distributed caching of data in web farms or implementing a distributed hash table (DHT).
We have a layer of web servers that handle the requests and a pool of machines that are used for caching. Each web server should be able to access any of the cache machines. The tricky aspect here is that when we cache some data as a result of a request to a particular web server, we should be able to retrieve that data from any of the other web servers.
Here is what we can do. We want to cache key value pairs and then at a later stage get hold of a cached value by providing the corresponding key. The code below takes a key and produces a hash bit array using SHA1. Then after some transformations we derive an integer number. A given key always produces the same number. In addition, all numbers are uniformly distributed which allows us to "page" them and assign them to a given number of cache machines or "buckets".
Thursday, April 21, 2016
ALGORYTHMS - animated and video examples
http://www.sorting-algorithms.com/
https://www.youtube.com/embed/vxENKlcs2Tw
https://www.youtube.com/embed/vxENKlcs2Tw
Tuesday, April 19, 2016
Wednesday, April 13, 2016
JAVASCRIPT - visualization of Bitcoins transactions by JS frameworks
Post in Twitter about the matter: https://twitter.com/pluralsight/status/719978158963978240
Example of JS visualization of Bitcoin transactions: http://bitbonkers.com/
JS framework Three JS: http://threejs.org/
JS framework Oimo JS: http://lo-th.github.io/Oimo.js/
Live stream of Bitcoins transactions: https://blockchain.info/
Example of JS visualization of Bitcoin transactions: http://bitbonkers.com/
JS framework Three JS: http://threejs.org/
JS framework Oimo JS: http://lo-th.github.io/Oimo.js/
Live stream of Bitcoins transactions: https://blockchain.info/
Tuesday, April 12, 2016
Monday, April 11, 2016
TCP - 3 Way handshake SYN, SYN-ACK, ACK
http://www.inetdaemon.com/tutorials/internet/tcp/3-way_handshake.shtml
| EVENT | DIAGRAM |
Host A sends a TCP SYNchronize packet to Host B
Host B receives A's SYN
Host B sends a SYNchronize-ACKnowledgement
Host A receives B's SYN-ACK
Host A sends ACKnowledge
Host B receives ACK.
TCP socket connection is ESTABLISHED.
|  TCP Three Way Handshake (SYN,SYN-ACK,ACK) |
Sunday, April 10, 2016
ORACLE - fundamentals of Relational model and Oracle RDBMS
Oracle officially Oracle is called an Object-relational database.
RDBMS - Relational Database Management System
Oracle v1 - 1978 and never released
Oracle 8i - 1999 i stands for Internet (possibility to query DB online)
Oracle 10g - 2000 g stands for Grid Computing
Oracle 12c - 2014 c stands for Cloud
Grid infrastructure refErs to deploying of a Massively Parallel infrastructure with storage fabric for building a shared storage that consist from pluggable hard drives (HDs) that can be extended by adding more HDs.
Automatic Storage Management (ASM) is Oracle software feature for Grid infrastructure.
Real Application Cluster (RAC) is an Oracle technology
In Oracle Grid infrastructure you can cluster your application servers, storage (ASM) and database (RAC) - and end-to-end high availability solution.
======== TWO MEMORY STRUCTURES IN ORACLE DB ==========================
SGA - System Global Area (main collection of shared data caches, for Instances - shareable memory)
PGA - Process Global Area (for each session of user, keeps its preferences, logs, SQL queries etc)
=========== ORACLE INSTANCE DOCS =================================
http://docs.oracle.com/database/121/CNCPT/startup.htm#CNCPT005
=========== PL/SQL anti-pattern - use of DML inside LOOP ==========
==================== ADVANTAGES OF ORACLE =======================
There are numerous features, that are quite unique. If I had to pick the ones that I consider killer, these would be:
Theory:
First description of Relational model in 1970 by EF "Ted" Codd: http://is.gd/eBKRHY
or http://bio.informatics.iupui.edu/beyond/misc/codd%20relational%20model%201970.pdf
Columns of the table could be called Domains
Rows of the table can be called Tuples
Table can be called Relation (that has n-tuples in it)
Active Domain of instant - the set of values represented at that instant.
Primary key - one domain of a given relation has values which uniquely identify each element (n-tuple) of that relation.
Primary key is nonredundant if it uniquely identify each element.
Relation can have more than one nonredundant primary keys, but only one of them should be selected and called THE primary key of this relation.
Foreign Key - domain of relation R is a Foreign Key if it is not the primary key of R but its elements are values of the primary key of some relation S.
Simple domains - domains whose elements are atomic (nondecomposable) values.
Normalization - is elimination of nonsimple domains.
Model - the relational view of data
Theory of Relations: https://en.wikipedia.org/wiki/Finitary_relation
Einstein's General relativity:
https://en.wikipedia.org/wiki/General_relativity
==================================
RDBMS - Relational Database Management System
Oracle v1 - 1978 and never released
Oracle 8i - 1999 i stands for Internet (possibility to query DB online)
Oracle 10g - 2000 g stands for Grid Computing
Oracle 12c - 2014 c stands for Cloud
Grid infrastructure refErs to deploying of a Massively Parallel infrastructure with storage fabric for building a shared storage that consist from pluggable hard drives (HDs) that can be extended by adding more HDs.
Automatic Storage Management (ASM) is Oracle software feature for Grid infrastructure.
Real Application Cluster (RAC) is an Oracle technology
In Oracle Grid infrastructure you can cluster your application servers, storage (ASM) and database (RAC) - and end-to-end high availability solution.
======== TWO MEMORY STRUCTURES IN ORACLE DB ==========================
SGA - System Global Area (main collection of shared data caches, for Instances - shareable memory)
PGA - Process Global Area (for each session of user, keeps its preferences, logs, SQL queries etc)
=========== ORACLE INSTANCE DOCS =================================
http://docs.oracle.com/database/121/CNCPT/startup.htm#CNCPT005
=========== PL/SQL anti-pattern - use of DML inside LOOP ==========
Never manipulate tables data (by DML) from inside PROCEDURE LOOPS.
Data Manipulation Language (DML) is a vocabulary used to retrieve and work with data in SQL Server 2016. Use these statements to add, modify, query, or remove data from a SQL Server database.https://apexapps.oracle.com/pls/apex/f?p=44785:141:0::NO::P141_PAGE_ID,P141_SECTION_ID:168,1208==================== ADVANTAGES OF ORACLE =======================
There are numerous features, that are quite unique. If I had to pick the ones that I consider killer, these would be:
- ASM (Automatic Storage Management) makes the storage management so much easier and smoother.
- RAC (Real Application Clusters) - this is something you will not find anywhere else and was the reason we went Oracle in the first place. Sure, there are replication or some sort of clustering solutions for other databases, but nothing comes close to RAC.
- Realiability - this is a good thing in a database :) Oracle just won't eat your data. I have seen MySQL databases corrupted beyond repair, I have seen MSSQL database fall to pieces. I have yet to see Oracle do something, that is not easily recoverable (with the right backup and HA strategy of course).
- Management - the Enterprise manager is awesome tool
- Monitoring and diagnostics - Oracle measures and reports everything and I mean everything. It is not always simple to extract, but all the information you need to tune or debug the database or applications using the database is available.
Theory:
First description of Relational model in 1970 by EF "Ted" Codd: http://is.gd/eBKRHY
or http://bio.informatics.iupui.edu/beyond/misc/codd%20relational%20model%201970.pdf
Columns of the table could be called Domains
Rows of the table can be called Tuples
Table can be called Relation (that has n-tuples in it)
Active Domain of instant - the set of values represented at that instant.
Primary key - one domain of a given relation has values which uniquely identify each element (n-tuple) of that relation.
Primary key is nonredundant if it uniquely identify each element.
Relation can have more than one nonredundant primary keys, but only one of them should be selected and called THE primary key of this relation.
Foreign Key - domain of relation R is a Foreign Key if it is not the primary key of R but its elements are values of the primary key of some relation S.
Simple domains - domains whose elements are atomic (nondecomposable) values.
Normalization - is elimination of nonsimple domains.
Model - the relational view of data
Theory of Relations: https://en.wikipedia.org/wiki/Finitary_relation
Mathematically, then, a relation is simply an "ordered set". (k-ary, e.g ternary = 3-ary)
When two objects, qualities, classes, or attributes, viewed together by the mind, are seen under some connexion, that connexion is called a relation.Newton's law of universal gravitation: https://en.wikipedia.org/wiki/Newton%27s_law_of_universal_gravitation
— Augustus De Morgan
Einstein's General relativity:
https://en.wikipedia.org/wiki/General_relativity
==================================
Friday, April 8, 2016
BIG DATA - MapReduce vs Massively Parallel Processing (MPP)
Comparison between MapReduce and MPP (Massively Parallel Processing)
Subscribe to:
Posts (Atom)