分类: Internet

  • HTTP Time Protocol

    Time sync over Http

    http://www.vervest.org/htp/

     

    INSTALL
    wget http://www.vervest.org/htp/archive/c/htpdate-1.1.3.tar.gz
    tar xvfz htpdate-1.1.3.tar.gz
    cd htpdate-1.1.3
    make

    SYNC EXAMPLE

    ./htpdate -P PROXY_IP:PROXY_PORT -s www.linux.org www.freebsd.org

    sudo ./htpdate -P PROXY_IP:PROXY_PORT -s www.linux.org www.freebsd.org

    sudo ./htpdate -P PROXY_IP:PROXY_PORT -s http://www.ntsc.ac.cn/ http://mrtg.synet.edu.cn/ http://www.baidu.com http://www.taobhao.com

  • Debian Dynamic DNS

    http://perceptionistruth.com/2013/05/running-your-own-dynamic-dns-service-on-debian/

    Running your own Dynamic DNS Service (on Debian)

    I used to have a static IP address on my home ADSL connection, but then I moved to BT Infinity, and they don’t provide that ability.  For whatever reason, my Infinity connection resets a few times a week, and it always results in a new IP address.

    Since I wanted to be able to connect to a service on my home IP address, I signed up to dyn.com and used their free service for a while, using a CNAME with my hosting provider (Gandi) so that I could use a single common host, in my own domain, and point it to the dynamic IP host and hence, dynamic IP address.

    While this works fine, I’ve had a few e-mails from dyn.com where either the update process hasn’t been enough to prevent the ’30 day account closure’ process, or in recent times, a mail saying they’re changing that and you now need to log in on the website once every 30 days to keep your account.

    I finally decided that since I run a couple of VPSs, and have good control over DNS via Gandi, I may as well run my own bind9 service and use the dynamic update feature to handle my own dynamic DNS needs.  Side note: I think Gandi do support DNS changes through their API, but I couldn’t get it working.  Also, I wanted something agnostic of my hosting provider in case I ever move DNS in future (I’m not planning to, since I like Gandi very much).

    The basic elements of this are,

    1. a bind9 service running somewhere, which can host the domain and accept the updates.
    2. delegation of a subdomain to that bind9 service.  Since Gandi runs my top level domain for me, I need to create a subdomain and delegate to it, and then make dynamic updates into that subdomain.  I can still use CNAMEs in the top level domain to hide the subdomain if I wish.
    3. configuration of the bind9 service to accept secure updates.
    4. a script to do the updates.

    In the interests of not re-inventing the wheel, I copied most of the activity from this post.  But I’ll summarise it here in case that ever goes away.

    Installing / Configuring bind9

    You’ll need somewhere to run a DNS (bind9 in my case) service.  This can’t be on the machine with the dynamic IP address for obvious reasons.  If you already have a DNS service somewhere, you can use that, but for me, I installed it on one of my Debian VPS machines.  This is of course trivial with Debian (I don’t use sudo, so you’ll need to be running as root to execute these commands),

    apt-get install bind9 bind9-doc

    If the machine you’ve installed bind9 onto has a firewall, don’t forget to open ports 53 (both TCP and UDP).  You now need to choose and configure your subdomain.  You’ll be creating a single zone, and allowing dynamic updates.

    The default config for bind9 on Debian is in /etc/bind, and that includes zone files.  However, dynamically updated zones need a journal file, and need to be modified by bind.  I didn’t even bother trying to put the file into /etc/bind, on the assumption bind won’t have write access, so instead, for dynamic zones, I decided to create them in /var/lib/bind.  I avoided /var/cache/bind because the cache directory, in theory, is for transient files that applications can recreate.  Since bind can’t recreate the zone file entirely, it’s not appropriate to store it there.

    I added this section to /etc/bind/named.conf.local,

    // Dynamic zone
      zone "home.example.com" {
        type master;
        file "/var/lib/bind/home.example.com";
        update-policy {
          // allow host to update themselves with a key having their own name
          grant *.home.example.com self home.example.com.;
        };
      };

    This sets up the basic entry for the master zone on this DNS server.

    Create Keys

    So I’ll be honest, I’m following this section mostly by rote from the article I linked.  I’m pretty sure I understand it, but just so you know.  There are a few ways of trusting dynamic updates, but since you’ll likely be making them from a host with a changing IP address, the best way is to use a shared secret.  That secret is then held on the server and used by the client to identify itself.  The configuration above allows hosts in the subdomain to update their own entry, if they have a key (shared secret) that matches the one on the server.  This stage creates those keys.

    This command creates two files.  One will be the server copy of the key file, and can contain multiple keys, the other will be a single file named after the host that we’re going to be updating, and needs to be moved to the host itself, for later use.

    ddns-confgen -r /dev/urandom -q -a hmac-md5 -k thehost.home.example.com -s thehost.home.example.com. | tee -a /etc/bind/home.example.com.keys > /etc/bind/key.thehost.home.example.com

    The files will both have the same content, and will look something like this,

    key "host.home.example.com" {
    algorithm hmac-md5;
    secret "somesetofrandomcharacters";
    };

    You should move the file key.thehost.home.example.com to the host which is going to be doing the updating.  You should also change the permissions on the home.example.com.keys file,

    chown root:bind /etc/bind/home.example.com.keys
    chmod u=rw,g=r,o= /etc/bind/home.example.com.keys

    You should now return to /etc/bind/named.conf.local and add this section (to use the new key you have created),

    // DDNS keys
    include "/etc/bind/home.example.com.keys";

    With all that done, you’re ready to create the empty zone.

    Creating the empty Zone

    The content of the zone file will vary, depending on what exactly you’re trying to achieve.  But this is the one I’m using.  This is created in /var/lib/bind/home.example.com,

    $ORIGIN .
    $TTL 300 ; 5 minutes
    home.example.com IN SOA nameserver.example.com. root.example.com. (
        1 ; serial
        3600 ; refresh (1 hour)
        600 ; retry (10 minutes)
        604800 ; expire (1 week)
        300 ; minimum (5 minutes)
        )
    NS nameserver.example.com.
    $ORIGIN home.example.com.

    In this case, namesever.example.com is the hostname of the server you’ve installed bind9 onto.  Unless you’re very careful, you shouldn’t add any static entries to this zone, because it’s always possible they’ll get overwritten, although of course, there’s no technical reason to prevent it.

    At this stage, you can recycle the bind9 instance (/etc/init.d/bind9 reload), and resolve any issues (I had plenty, thanks to terrible typos and a bad memory).

    Delegation

    You can now test your nameserver to make sure it responds to queries about the home.example.com domain.  In order to properly integrate it though, you’ll need to delegate the zone to it, from the nameserver which handles example.com.  With Gandi, this was as simple as adding the necessary NS entry to the top level zone.  Obviously, I only have a single DNS server handling this dynamic zone, and that’s a risk, so you’ll need to set up some secondaries, but that’s outside the scope of this post.  Once you’ve done the delegation, you can try doing lookups from anywhere on the Internet, to ensure you can get (for example) the SOA for home.example.com.

    Making Updates

    You’re now able to update the target nameserver, from your source host using the nsupdate command.  By telling it where your key is (-k filename), and then passing it commands you can make changes to the zone.  I’m using exactly the same format presented in the original article I linked above.

    cat <<EOF | nsupdate -k /path/to/key.thehost.home.example.com
    server nameserver.example.com
    zone home.example.com.
    update delete thehost.home.example.com.
    update add thehost.home.example.com. 60 A 192.168.0.1
    update add thehost.home.example.com. 60 TXT "Updated on $(date)"
    send
    EOF

    Obviously, you can change the TTL’s to something other than 60 if you prefer.

    Automating Updates

    The last stage, is automating updates, so that when your local IP address changes, you can update the relevant DNS server.  There are a myriad ways of doing this.  I’ve opted for a simple shell script which I’ll run every couple of minutes via cron, and have it check and update DNS if required.  In my instance, my public IP address is behind a NAT router, so I can’t just look at a local interface, and so I’m using dig to get my IP address from the opendns service.

    This is my first stab at the script, and it’s absolutely a work in progress (it’s too noisy at the moment for example),

    #!/bin/sh

    # set some variables
    host=thehost
    zone=home.example.com
    dnsserver=nameserver.example.com
    keyfile=/home/bob/conf/key.$host.$zone
    #

    # get current external address
    ext_ip=`dig +short @resolver1.opendns.com myip.opendns.com`

    # get last ip address from the DNS server
    last_ip=`dig +short @$dnsserver $host.$zone`

    if [ ! -z “$ext_ip” ]; then
    if [ ! -z “$last_ip” ]; then
    if [ “$ext_ip” != “$last_ip” ]; then
    echo “IP addresses do not match (external=$ext_ip, last=$last_ip), sending an update”

    cat <

    http://www.foell.org/justin/diy-dynamic-dns-with-openwrt-bind/

    http://blog.infertux.com/2012/11/25/your-own-dynamic-dns/

    http://idefix.net/~koos/dyndnshowto/dyndnshowto.html

    https://blog.hqcodeshop.fi/archives/76-Doing-secure-dynamic-DNS-updates-with-BIND.html

    https://0x2c.org/rfc2136-ddns-bind-dnssec-for-home-router-dynamic-dns/

    http://agiletesting.blogspot.com/2014/12/dynamic-dns-updates-with-nsupdate-new.html

     

     

  • dnstap

    http://dnstap.info/slides/dnstap_nanog61.pdf

     

    dnstap

     –  What  is  it?

    • High  speed  DNS  logging  without  packet  capture
    • Encoding  uses  Protocol  Buffers
    • Binary  clean
    • Efficient  encoding
    • Extendable
    • Implementa6ons  available  for  many  programming
  • HTTP PROXY 协议

    https://imququ.com/post/web-proxy.html

     

    HTTP 代理存在两种形式,分别简单介绍如下:

    第一种是 RFC 7230 – HTTP/1.1: Message Syntax and Routing(即修订后的 RFC 2616,HTTP/1.1 协议的第一部分)描述的普通代理。这种代理扮演的是「中间人」角色,对于连接到它的客户端来说,它是服务端;对于要连接的服务端来说,它是客户端。它就负责在两端之间来回传送 HTTP 报文。

    第二种是 Tunneling TCP based protocols through Web proxy servers(通 过 Web 代理服务器用隧道方式传输基于 TCP 的协议)描述的隧道代理。它通过 HTTP 协议正文部分(Body)完成通讯,以 HTTP 的方式实现任意基于 TCP 的应用层协议代理。这种代理使用 HTTP 的 CONNECT 方法建立连接,但 CONNECT 最开始并不是 RFC 2616 – HTTP/1.1 的一部分,直到 2014 年发布的 HTTP/1.1 修订版中,才增加了对 CONNECT 及隧道代理的描述,详见 RFC 7231 – HTTP/1.1: Semantics and Content。实际上这种代理早就被广泛实现。

  • 嵌入式web服务器

    转:http://www.cnblogs.com/xmphoenix/archive/2011/04/12/2013394.html

    boa、thttpd、mini_httpd、shttpd、lighttpd、goaheand、appweb

     

    一个网友的个人意见:

    boa 的功能比较齐全, 便对嵌入式应用很多功能就是冗余(如virtual host), 内存使用量较大些.
    thttpd 功能较少, 实现简单. 内存使用量较少. 同时比较方便扩展.
    shttpd 功能功能算是比较全的, 但在处理二进制数据时不够稳定, 时有异常. 有待观察.
    light-httpd, apache 属重量级服务器, 成熟稳定, 体积较大, 在复杂的嵌入式应用上可选用.
    goAhead 是个比较专用的 webserver, 大部分功能都在服务它自己提供的 goform 功能和
    ASP/javascript 功能. 最后的 2.1.8 版仍有不少bug. (见下)
    mini-httpd 与 thttpd 是同一家, 功能几乎完全一样.
    boa 缺陷:
    (1) 未提供 CGI 解析头处理.
    可按这个地址方便修改. http://bbs.chinaunix.net/viewthread.php?tid=824840
    (2) 对 POST 数据使用临时文件缓冲, 对无法创临时文件的小系统系统, 需要手工改下这部代码.
    很多人报告在移植时不能POST 数据, 都是这个原因.
    (3) …
    thttpd 缺陷:
    (1)  CGI1.1 标准支持不完整(不般影响不大), 未提供对协议要求的其它HTTP头处理,
    如:If-Modified-Since, Accept-Language等应用程序就收不到.
    (2) 直接使用 socket 到 CGI 应用的重定, 会导致提供大量 POST 数据时(如上传文件),
    CGI应用不读完全部 POST 数据就无法向浏览器应答 bug
    (3) …
    goAhead 缺陷:
    (1) 专用, 如喜欢它提供的 goform和 asp 令论.
    (2) CGI 对二进制输出有很多 bug.
    (3) 为实现单一任务处理, 在很平台采用延时轮询接收队列, 处理效率不高.
    (4) 其它 bug 有不一罗列了, 移植时要一个个订下.

  • libnet-dev

    http://sourceforge.net/projects/libnet-dev/files/?source=navbar

    A portable framework for low-level network packet construction

  • Best C/C++ Network Library

    http://stackoverflow.com/questions/118945/best-c-c-network-library

    Aggregated List of Libraries

  • Update NO-IP DDNS

    Updating no-ip ddns on

    DDWRT Router:  

    with the help of inadyn tool

    https://github.com/torglobit/inadyn

     

     

     

    Linux box:

    With the help of noip-udc-linux

    http://www.no-ip.com/client/linux

     

    Failed Issus:

    No direct internet link

       DNS resolve too slow

    inadyn:   timeout=IP_DEFAULT_TIMEOUT=20s

    noip-udc:   using gethostbyname  linux api to get IP address,  the timeout can be defined in /etc/resolv.conf,  the default value in linux is 5 ms,

    Example of resolv.conf:

    nameserver 8.8.8.8 options timeout:30