Sunday, January 26, 2020

父亲



因为我爷爷去世的比较早,父亲作为家里的长兄,很早就担负起了家庭的重担,在学习之余,父亲还去火车站做临时搬运工,扛大包,赚钱来补贴家用。我对父亲的工作内容并不太了解,但回想起来,印象中有几件事让我对父亲充满的敬佩。八十年代初,秦岭发生了特大洪水,我父亲一直坚守在抗洪抢险第一线,直到膝盖负伤才回来休息。八十年代末,父亲又为了援助建设突尼斯水电工程,外派出差两年之久,把中国的水利水电技术带到北非。父亲在退休前有段时间,经常上山下乡去指导农村的小水电建设,长期的山路颠簸也引发了他椎间盘突出的毛病。

父亲对家庭也倾注了很大的心血。我们的家庭是严父慈母类型。父亲的话不多,君子纳于言而敏于行,父亲以他的行动指导了我姐和我的人生准则,使我们终生受益。父母的爱是和风细雨的,从日常生活的一点一滴体现出来。在我们高中阶段,为了让我姐和我专心学习,每天中午父亲都要骑车赶回家给我们做午饭。有一次我在回家路上看到他急匆匆地骑车往家赶,他并没有看到我,但我到现在还记得那份感动。还有一段美好的记忆如此生动地刻在我的脑海里。那是一个风和日丽的初夏休息日,我父亲三十多岁,风华正茂,骑着他的自行车,前面带着我,后面坐着我姐,从清姜大坡飞一般地骑下来。

Thursday, January 31, 2019

Hadoop and Spark Setup

#get centos container
git pull centos

#start centos with port mapping
docker run -d -p 8088:8088 -p 9870:9870 --name "centos" -i centos

#access centos
docker exec -it centos /bin/bash

#download the instructions
wget https://www.clustermonkey.net/download/LiveLessons/Hadoop_Fundamentals/Hadoop_Spark_Fundamentals_Code_Notes-V3.0.tgz
cd /tmp
tar xzvf Hadoop_Spark_Fundamentals_Code_Notes-V3.0.tgz

#follow the instructions in /tmp/Hadoop_Spark_Fundamentals_Code_Notes-V3.0/Lesson-2/Lesson-2.2/NOTES.txt to setup hadoop, pig, hive

a few catches
* adjust the JAVA_HOME and HADOOP_HOME accordingly based on your installation
* modify HADOOP_PATH in the script hadoop-setup-script.sh under /tmp/Hadoop_Spark_Fundamentals_Code_Notes-V3.0/Lesson-2/Lesson-2.2/scripts
* check if logs under /opt/hadoop-3.2.0/ is created
* after hadoop 3.0.0, the namenode port is 9870 instead of 50070
* before running yarn, set the YARN_CONF_LIB
export YARN_CONF_LIB=/opt/hadoop-3.2.0/etc/hadoop


##follow the instructions in /tmp/Hadoop_Spark_Fundamentals_Code_Notes-V3.0/Lesson-2/Lesson-2.3/NOTES.txt to setup spark


Sunday, May 27, 2018

More Principles - Clean Architecture: A Craftsman's Guide to Software Structure and Design, First Edition

REP: Reuse/Release Equivalence Principle, the granule of reuse is the granule of release
CCP: Common Closure Principle
CRP: Common Reuse Principle, don't force to depend on things they don't need
ADP: Acyclic Dependencies Principle, no cycles in the dependency graph
SDP: Stable Dependencies Principle, depend in the direction of stability
SAP: Stable Abstractions Principle, as abstract as stable

S.O.L.I.D

SRP: Single Responsibility Principle, a module should be responsible to one and only one actor
OCP: Open-Closed Principle, a software artifact should be open for extension but closed for modification
LSP: Liskov Substitution Principle, interchangeable parts adhere to a contract
ISP: Interface Segregation Principle, avoid depending on things not use
DIP: Dependency Inversion Principle, details should depend on policies

Thursday, March 01, 2018

generate pem from keypair, and sign message

openssl rsa -in id_rsa -outform pem > id_rsa.pem
openssl rsa -in id_rsa -pubout -out id_rsa.pub.pem
openssl dgst -sha256 -sign ~/.ssh/id_rsa.pem -out signature.data message
openssl dgst -sha256 -verify ~/.ssh/id_rsa.pub.pem -signature signature.data message

start the ethereum private net on windows



download/install geth from https://github.com/ethereum/go-ethereum/wiki/Installation-instructions-for-Windows

setup private net jason file under ~/.privether, an example of privategenesis.json

{
  "nonce": "0x00006d6f7264656e",
  "difficulty": "0x000002",
  "mixhash": "0x00000000000000000000000000000000000000647572616c65787365646c6578",
  "coinbase": "0x0000000000000000000000000000000000000000",
  "timestamp": "0x00",
  "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "extraData": "0x",
  "gasLimit": "0x2FEFD8",
  "config": {
        "chainId": 15,
        "homesteadBlock": 0,
        "eip155Block": 0,
        "eip158Block": 0
  },
  "alloc": {
    "cd2a3d9f938e13cd947ec05abc7fe734df8dd826": { "balance": "10000000000000000" }
  }
}

initialize the private net and specify a network id

geth --datadir ~/.ethereum/privatenet init .privether/privategenesis.json
geth --datadir ~/.ethereum/privatenet --networkid=346

in another console, attach to this private net

geth attach ipc:\\.\pipe\geth.ipc

and happy mining!


> personal.newAccount("Password123")
"0x01e08a06618d5a96956080ead0c8fc2452cf3311"
> miner.setEtherbase(personal.listAccounts[0])
true
> eth.getBalance(eth.coinbase).toNumber()
0
> miner.start(2)
null
> eth.getBalance(eth.coinbase).toNumber()
1.415e+21


Sunday, October 30, 2016

install dc/os on centos 7

first install and start docker
sudo yum -y install docker docker-registry
[osboxes@osboxes ~]$ sudo systemctl enable docker.service
[osboxes@osboxes ~]$ sudo systemctl start docker.service
[osboxes@osboxes ~]$ sudo systemctl status docker.service


install based on https://dcos.io/docs/1.8/administration/installing/custom/gui/

if you meet permission issue, you need disable selinux

[osboxes@osboxes ~]$ cat /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted







父亲

因为我爷爷去世的比较早,父亲作为家里的长兄,很早就担负起了家庭的重担,在学习之余,父亲还去火车站做临时搬运工,扛大包,赚钱来补贴家用。我对父亲的工作内容并不太了解,但回想起来,印象中有几件事让我对父亲充满的敬佩。八十年代初,秦岭发生了特大洪水,我父亲一直坚守在抗洪抢险第一...