Hikvision Raptor Tech Notes

[CBX 0]

I had to recently set DST as here in the UK we've just gone to BST (British Summer Time).

 

Set DST on the cameras so the OSD is correct, but couldn't understand why my automated video and picture extract script was producing filenames an hour out (their filenames include the time). Hikvision uses linux (POSIX) timestamps which shouldn't care about timezone, but they haven't implemented it correctly in the binary files created to store videos and pictures, such that it's not seconds since 1 Jan 1970 UTC (ignoring leap seconds) but rather what timezone you set on your camera.

 

It could be I'm wrong re the above as I live in a country either UTC+0 or UTC+1 so it's less obvious, but it's the conclusion I've come to.

 

As such I've had to hard code the following into my scripts:

 

//Override possible BST as Hikvision stores timestamps wrongly
date_default_timezone_set('UTC');

 

 

Annoying and while the average user won't come across this thought it was worth putting here.

 

If I'm wrong please let me know

[CBX 0]

I wrote a script to automatically extract my motion detection videos from the cameras, but given that they have a 5 second pre-record period, it gets annoying to review all of them when some of them are unavoidably weather or due to car headlights etc.

 

So I've written a script to use ffmpeg/avconv to capture a still from 8 secs in each motion detection event, and then create a video of these stills (each snapshot displayed for 2 seconds each), and then I only need to look at 1 vid per camera per day to get a sense of what happened. I have these linked to an XBMC keyboard shortcut and can then always review the full unedited motion detection video specific to the situation if needed.

 

e.g. (create still 8 seconds in):

 

/usr/bin/avconv -i /mnt/hdd/sd1/12-04-14/17_32___12-04-14_17_32_41_to_12-04-14_17_32_57.mp4 -r 25.0 -frames:v 1 -ss 00:00:08 /mnt/hdd/sd1/_motpic/12-04-14/17_32___12-04-14_17_32_41_to_12-04-14_17_32_57.jpg

 

After linking the snapshot pics of the action to a sequential file list, create overview video:

 

/usr/bin/avconv -r 1/2 -y -i /tmp/4641/temp%03d.jpg -r 30 /mnt/hdd/sd1/_motpic/12-04-14/12-04-14.mp4

 

Then look at the created summary file (e.g. /mnt/hdd/sd1/_motpi/12-04-14/12-04-14.mp4) and you know what happened that day, if it requires your attention and it will only take 20 seconds for 10 events (or 10 seconds if the display per snapshot is 1 second etc).

[Mox 0]

Hi CBX,

 

you wrote a few posts earlier about "ONVIF monitoring" for triggering a script when a motion is detected. It would be great if you could share the code.

 

Many Thanks!

[CBX 0]

Though I'm not publishing my code at present I can say I based it on the below php ONVIF class written by someone else:

 

http://www.phpclasses.org/package/7996-PHP-Control-network-video-devices-with-ONVIF-protocol.html

 

He's got a fair few methods for PTZ operations which do not apply to me, so you can always rip those out.

 

Also I'd recommend wireshark to find out the XML comms between your camera and the control app!

[patsy 0]

CBX,

 

I just purchased a DS-2CD2432F-I (no 'W'). Yes, no wireless, but interestingly, the wireless module and antenna are on the board.

 

So, I assume the hardware is the same between the DS-2CD2432F-I and the DS-2CD2432F-IW, which probably means that the manufacturer is just changing the firmware, much like how it makes Chinese and English versions of the cameras.

 

I assume it might be possible to do a patch similar to the Chinese day of the week fix to make the wireless work. Does that seem like it would work?

 

Anyway, if you'd like to investigate this, great. If not, I'll get to it as my interest waxes and wanes.

 

Obviously there are ethical issues with actually making the wireless work on a device sold without it enabled. Personally, I do not intend to benefit from this- I just want to see if it could be made to work (but I really prefer running it wired), but I understand that others may profit off of it, causing the manufacturer lost profits.

 

I assume this is why a there is no permanent fix for the Chinese day of the week. Though there might be checksum issues in making a fix in the flash, they probably are not insurmountable.

[CBX 0]

Certainly if the hardware is all there then a patch (or changing values in the flash) would work for wireless.

 

I myself have no ethical issues with this as I consider that if you bought a product which has been crippled by the manufacturer then you have the right to modify your own property. If you were selling them then that would be a different case.

 

I don't have that camera, and like you say wired is preferred for most. But you could likely come up with a patch if so minded. I think I activated wireless on mine when playing with the code (though mine don't have the hardware of course).

 

Changing the flash is not insurmountable, it just comes with risks of bricking your camera and future detection by Hikvision if they decide to check in future firmware updates (though I suppose you could revert your flash values back before flashing or investigate new davinci first). I think I could do it, but I don't want to risk my cameras and given I modify them so much anyway am happy with patching davinci in memory before execution.

[CBX 0]

Built and applied custom firmware:

 

http://www.cctvforum.com/viewtopic.php?t=41103

[esyn 0]

Hi!

 

I see you've done a great job on hacking your cam. I've bought a DS-2CD2012-I recently, and I'm pretty happy with it. Except that I can't find any convenient way to play recorded videos under Linux. At first I thought I could play them using RTSP , but It doesn't seem to work. And I've got no reply from Hikvision support regarding this either. I wonder if this can be fixed, but right now I don't know where to start.

 

After this had failed, I decided to extract videos out of NFS storage. I've written a script that does this, and I see you've done the same. What I wanted to ask is this: could you share what you know about the format of their index file? I can share what I've found out too, but it's really a minimum number of fields required to extract, and you most likely already know what I know. I think that maybe I could improve my solution if I knew the format better. Later I plan on building a simple web interface to browse the extracted videos.

 

Keep up the good work!

[swimma 0]

Hello guys, I appreciate you work! Would you be so nice to share your scripts for video extraction with the community? Keep up the good work! Alex

[esyn 0]

Hi, here are mine. They are not polished or bug-free, of course, but might be helpful to someone.

 

https://github.com/iafilatov/hikstract

https://github.com/iafilatov/hikweb

 

Unfortunately, I received no notification of your post. Would've posted this sooner otherwise…

[swimma 0]

I found this solution written in C in the meantime:

 

https://github.com/aloz77/hiktools

[kidd 0]

This will likely interest no one but me, but in the course of working on the camera, I've been annoyed by the lack of functionality with the installed tools (though in fairness to Hikvision they've put what's needed for most people).

 

So I've cross compiled a new busybox to replace the old one.

 

Original functions:

 

Currently defined functions:
       ash, awk, bash, cat, chmod, cp, date, dd, df, dmesg, du, echo,
       env, false, fdisk, free, fsync, ftpget, ftpput, gunzip, gzip, halt,
       hwclock, ifconfig, inetd, init, insmod, iostat, kill, linuxrc, ln,
       login, ls, lsmod, mdev, mkdir, mkdosfs, mkfs.vfat, mknod, modprobe,
       mount, mpstat, mv, nanddump, nandwrite, netstat, ping, ping6, poweroff,
       ps, pwd, reboot, rm, rmmod, route, sed, sh, sleep, sync, tail, tar,
       telnetd, test, tftp, top, touch, true, umount, zcat

 

My one with more tools:

 

        acpid, add-shell, addgroup, adduser, adjtimex, arp, arping,
       ash, awk, base64, basename, beep, blkid, blockdev, bootchartd, brctl,
       bunzip2, bzcat, bzip2, cal, cat, catv, chat, chattr, chgrp, chmod,
       chown, chpasswd, chpst, chroot, chrt, chvt, cksum, clear, cmp, comm,
       cp, cpio, crond, crontab, cryptpw, cttyhack, cut, date, dc, dd,
       deallocvt, delgroup, deluser, depmod, devmem, df, diff, dirname,
       dmesg, dnsd, dnsdomainname, dos2unix, du, dumpkmap, echo, ed, egrep,
       env, envdir, envuidgid, ether-wake, expand, expr, fakeidentd, false,
       fbset, fbsplash, fdflush, fdformat, fdisk, fgconsole, fgrep, find,
       findfs, flash_lock, flash_unlock, flashcp, flock, fold, free,
       freeramdisk, fsck, fsck.minix, fsync, ftpd, ftpget, ftpput, fuser,
       getopt, getty, grep, groups, gunzip, gzip, halt, hd, hdparm, head,
       hexdump, hostid, hostname, hush, hwclock, id, init, insmod, install,
       ionice, iostat, ipcrm, ipcs, kbd_mode, kill, killall, killall5,
       klogd, last, less, linux32, linux64, linuxrc, ln, loadfont, loadkmap,
       logger, login, logname, logread, losetup, ls, lsattr, lsmod, lspci,
       lsusb, lzcat, lzma, lzop, lzopcat, makedevs, man, md5sum, mdev, mesg,
       microcom, mkdir, mkdosfs, mke2fs, mkfifo, mkfs.ext2, mkfs.minix,
       mkfs.vfat, mknod, mkpasswd, mkswap, mktemp, modinfo, modprobe, more,
       mount, mountpoint, mpstat, mt, mv, nameif, nbd-client, nc, nice,
       nmeter, nohup, od, openvt, passwd, patch, pgrep, pidof, ping, ping6,
       pipe_progress, pivot_root, pkill, pmap, poweroff, powertop, printenv,
       printf, ps, pscan, pstree, pwd, pwdx, raidautorun, rdate, rdev,
       readahead, readlink, readprofile, realpath, reboot, remove-shell,
       renice, reset, resize, rev, rm, rmdir, rmmod, rpm, rpm2cpio, rtcwake,
       run-parts, runlevel, runsv, runsvdir, rx, script, scriptreplay, sed,
       seq, setarch, setconsole, setfont, setkeycodes, setlogcons,
       setserial, setsid, setuidgid, sh, sha1sum, sha256sum, sha512sum,
       showkey, slattach, sleep, smemcap, softlimit, sort, split,
       start-stop-daemon, stat, strings, stty, su, sulogin, sum, sv, svlogd,
       swapoff, swapon, switch_root, sync, sysctl, syslogd, tac, tail, tar,
       tcpsvd, tee, telnet, test, time, timeout, top, touch, tr, traceroute,
       true, tty, ttysize, tunctl, tune2fs, ubiattach, ubidetach, ubimkvol,
       ubirmvol, ubirsvol, ubiupdatevol, udpsvd, umount, uname, unexpand,
       uniq, unix2dos, unlzma, unlzop, unxz, unzip, uptime, users, usleep,
       uudecode, uuencode, vconfig, vi, vlock, volname, wall, watch,
       watchdog, wc, wget, which, who, whoami, whois, xargs, xz, xzcat, yes,
       zcat, zcip

 

I've put mine in /dav/opt and then run a script on boot to link functions which don't already exist:

 

/ # cat /dav/opt/busybox.sh
for FUNC in `/dav/opt/busybox --list-functions`; do
 ln -s /dav/opt/busybox /"$FUNC"
done

 

This gives things like syslogd and klogd as well as things I use a lot like vi, grep, strings, less, md5sum etc.

 

You can also use busybox to run a full http, smtp, pop server etc on the camera but i don't need those. Being about to telnet from the camera to other places has proved helpful though.

 

Great job, CBX

can you share your busybox?

 

Thanks

[telc 0]

Is there anyway to setup a cron on the stock hikvision firmware? I would like to setup a cron to run once a week and reboot the camera. For some reason after a week the liveview gets choppy when there is motion detection, a reboot always fixes it and its good for several more days.

[monitor2015 0]

Hi all

 

Hi. I forgot admin password and need to recover my DVR

Please help me.

My Device Info

 

Device Serial No: DS-7216HVI-ST0120120913AAWR411138590WCVU

Start Time: 2015-04-06 08:27:00

 

With the best regards.

[bearclaw 0]

Hi. I forgot admin password and need to recover my DVR

Please help me.

My Device Info

 

Device Serial No: DS-7216HVI-ST0120120913AAWR411138590WCVU

Start Time: 2015-04-06 08:27:00

 

With the best regards.

 

http://www.ipcamtalk.com/showthread.php/2920-Hikvision-camera-admin-password-reset-tool

[goldserve 0]

I found this solution written in C in the meantime:

 

https://github.com/aloz77/hiktools

 

I compiled this program in freebsd on my freenas server but I get the following error:

 

error: logger.main - Can't read 80 bytes for segment index

 

Any ideas?

[swimma 0]

error: logger.main - Can't read 80 bytes for segment index

Any ideas?

 

How do you run the program? If you do it like this

 

./extract_video -i /media/ipcam/datadir0/ -o /path/to/output -l

 

it's expecting the index file /media/ipcam/datadir0/index00.bin to contain the video index. Does it?

[goldserve 0]

error: logger.main - Can't read 80 bytes for segment index

Any ideas?

 

How do you run the program? If you do it like this

 

./extract_video -i /media/ipcam/datadir0/ -o /path/to/output -l

 

it's expecting the index file /media/ipcam/datadir0/index00.bin to contain the video index. Does it?

 

 

I run it exactly the way you said and I also put in -v for verbose.

 

FILE 146 =================
fileNo: 119495468
chan: 28327
startTime: 109836406-05-25 03:41:51
endTime: 1970-01-01 00:00:00
FILE 147 =================
fileNo: 268394496
chan: 22168
startTime: 5645-07-10 22:22:05
endTime: 111197427-05-09 10:49:27
FILE 148 =================
fileNo: 122015904
chan: 37418
startTime: 111197427-05-09 10:49:27
endTime: 1970-01-01 00:00:00
error: logger.main - Can't read 80 bytes for segment index

 

Looks like either the file structure changed a little and everything is fubar'd now. I can run the python script and that extracts the data fine.

[goldserve 0]

Time for me to give back to the community! I modified and fixed the python scripts to extract videos, create a snapshot of pictures at 5.5seconds into the video and create a new !summary video from the pictures. All I have to do now is to review the !summary.mp4 video everyday and this runs on my freenas server, inside a jail with lang/python34, multimedia/libav, and multimedia/ffmpeg installed.

 

http://1drv.ms/1EJVC8Z

 

In freenas, create a new jail based on busy box template.

Add storage to that contains the folder to your camera's NFS storage

Download the files and extract to some directory in the jail

Edit the paths in config.cfg

execute the following in the jail and accept everything:

pkg install lang/python34

pkg install multimedia/libav

pkg install multimedia/ffmpeg

Make extracted python files executable (chmod +x *)

run ./hikstract

 

Enjoy!

[dgshue 0]

I'm searching for some assistance "parsing" the index files, specifically the index00p.bin to collect metadata from the *.pic files. It looks like CBX was successful in identifying the format of the binary file to do it with his 'extract_hik_picindex.php'. Unfortunately, it doesn't appear that he shared that with anyone on the list. I'm working in Perl and so far I've been successful in working with the index00.bin (video index), but my goal here is extracting the date/time stamp from the index00p.bin for the captured snapshots. I cannot find the binary file format that has been described in others work that I've identified on GitHub. Does anyone know what the format is and/or have a copy of CBX's php code so that I determine the format of index00p.bin?

 

TIA

 

 

 

Video and picture extraction

 

You can parse the video and picture indexes on your NAS or SD card, and extract videos and pictures. Of course, you can do this manually on your camera via the web interface, but I prefer for this to happen automatically so I extract from my SD card to my primary server which has 5Tb+ rather than 32Gb

 

I then generate a playlist for xbmc with last nights motion detection recordings.

 

root@rl70:/# /usr/local/bin/extract_hik_picindex.php --indexfile=/mnt/sd2/datadir0/index00p.bin  --extract-pics=yes  --extract-dir=/mnt/hdd/sd2pic
Start Time: 24-02-14 19:17:55
End Time: 25-02-14 11:02:59
Number of extents: 1
Number of pics: 968
Total pic size: 135296848
Extracting to /mnt/hdd/sd2pic/25-02-14/11_00_58.jpg .... OK
Extracting to /mnt/hdd/sd2pic/25-02-14/11_01_58.jpg .... OK
Extracting to /mnt/hdd/sd2pic/25-02-14/11_02_58.jpg .... OK
3 pics extracted. 

 

Videos (the extracted filenames look user unfriendly but suit my needs!).

 

root@rl70:/# /usr/local/bin/extract_hik_vidindex.php  --indexfile=/mnt/sd2/datadir0/index00.bin --type=Motion --extract-videos=yes --extract-dir=/mnt/hdd/sd2
Number of videos: 5
Total video size: 10325437836
File /mnt/hdd/sd2/24-02-14/20_57___24-02-14_20_57_56_to_24-02-14_20_58_16.mp4 exists - skpping....
File /mnt/hdd/sd2/24-02-14/22_24___24-02-14_22_24_20_to_24-02-14_22_24_38.mp4 exists - skpping....
File /mnt/hdd/sd2/24-02-14/23_20___24-02-14_23_20_03_to_24-02-14_23_20_23.mp4 exists - skpping....
File /mnt/hdd/sd2/24-02-14/23_21___24-02-14_23_21_41_to_24-02-14_23_22_01.mp4 exists - skpping....
File /mnt/hdd/sd2/25-02-14/06_38___25-02-14_06_38_39_to_25-02-14_06_38_59.mp4 exists - skpping....
File /mnt/hdd/sd2/25-02-14/10_03___25-02-14_10_03_30_to_25-02-14_10_03_48.mp4 exists - skpping....
0 videos extracted.

 

Here's an example of a video extent and video object:

 

    [18] => HIK_EXTENT_INDEX Object
       (
           [index_num] => 18
           [video_count] => 2
           [time_start] => 1393067869
           [time_start_str] => 22-02-14 11:17:49
           [time_end] => 1393090333
           [time_end_str] => 22-02-14 17:32:13
           [duration] => 22464
           [data1] => 8308
           [data2] => 17927
       )

   [21-02-14_16_38_14_to_21-02-14_17_05_51] => HIK_VIDEO_INDEX Object
       (
           [video_id] => 21-02-14_16_38_14_to_21-02-14_17_05_51
           [video_extent_filename] => /mnt/sd2/datadir0/hiv00032.mp4
           [video_extract_filename] => /mnt/hdd/sd2/21-02-14/16_38___21-02-14
_16_38_14_to_21-02-14_17_05_51.mp4
           [video_extent_id] => 32
           [video_start_pos] => 00000000
           [video_end_pos] => 0f873828
           [video_start_pos_dec] => 0
           [video_end_pos_dec] => 260519976
           [video_size] => 260519976
           [video_type_int] => 00
           [video_type_str] => Continuous
           [video_time1] => 1393000694
           [video_time2] => 1393002351
           [video_time3] => 1393000694
           [video_time4] => 1393000694
           [video_time5] => 1393002351
           [video_start_str] => 21-02-14 16:38:14
           [video_end_str] => 21-02-14 17:05:51
           [video_duration] => 1657
           [video_data1] => 000c6f98
           [video_data2] => 000399e0
           [video_data3] => 000c6f98
           [video_data4] => 26a9b673
           [video_data5] => 2b1b1657
           [video_data6] => 0000
           [video_data7] => 4aa8
       )

 

I'm still working on some of the extra data which is to do with frames etc.