Tag: code
- Posted: 2 years ago
- Tags:code, flv, ruby
There is a lack of decent open source FLV manipulation software out there. There is Yamdi which is a seriously awesome metadata injector, written in C, and then there is FLVTool2, which is a ruby based injector tool. FLVTool2 is really awesome, but it is a serious memory hog and 90% of the files I work with are >1gb size.
I am doing some house cleaning on one project and I have a need to quickly get the duration of an flv video file, but without loading the whole thing into memory, or even reading in the whole file. As an advantage to me, all the files in question have already been injected by yamdi, so the duration is already calculated and inserted into the files. I just need to read it, and be on my merry way.
Here it is:
def duration(filename)
f = File.open(filename, 'r')
raise "Could not open File!" unless f
#check for flv text
unless f.read(3) === 'FLV'
raise "This does not seem to be an FLV file."
end
#check to make sure we are a video
f.seek(3)
is_video = f.read(1).unpack("H*").first.hex
unless is_video == 1
raise "This FLV file does not contain video."
end
#seek to end, get size
f.seek(0, IO::SEEK_END)
length = f.tell
#calculate tag length
f.seek(-4, IO::SEEK_END)
taglen = f.read(4).unpack("H*").first.hex
#finally get duration
f.seek(length - taglen, IO::SEEK_SET)
duration = f.read(3).unpack("H*").first.hex.to_f/1000
f.close()
#duration returns in seconds
duration
end
I ripped this out of another one of my classes that is already dealing with the flv file in other ways, so there is more validation in place than you see here. This has only been tested with files injected with yamdi 1.3 and 1.4, I have not tested with anything else, but if anyone does please let me know!
If you are interested in doing this in php there is similar scripts
here which helped me greatly.
- Posted: 2 years ago
- Tags:code, productivity, redmine, twitter
What do you do when you have mail, instant messages, rss feeds, web pages, text messages and revision control to all keep up with? Myneid came up with an idea of getting redmine updates from within twitter, so I took a crack at it:
#!/usr/bin/env ruby
require 'rubygems'
gem 'feedtools'
gem 'htmltokenizer'
require 'feed_tools'
require 'html/htmltokenizer'
require 'twitter'
#your redmine url
feed_url = "http://redmine.yourcompany.net/projects/activity?format=atom&key=YOURRSSKEY"
#grab redmine data
feed = FeedTools::Feed.open(feed_url)
post_these = Array.new
#fetch salient pieces from this stream
feed.items.each do |act|
tweet = "#{act.title}"
t = HTMLTokenizer.new(act.description)
desc = String.new
unless ARGV[0].nil? or ARGV[0] != 'full'
while token = t.getTag('p')
desc << t.getTrimmedText('p').gsub(/\n/, ' ')
end
tweet << "\t#{desc[0..137]}..."
end
if act.time >= 5.minutes.ago
post_these << tweet
end
end
#connect to twitter if we need to
if post_these.size > 0
twitter = Twitter::Base.new('YOURTWITTERUSERNAME', 'YOURTWITTERPASSWORD')
end
#post the oldest first
for tweet in post_these.reverse
status = twitter.post(tweet)
end
Beware, this script has no real error checking, but as long as your credentials and access key are all correct it should be golden. You will need the feedtools, twitter, and html tokenizer gems to get it to run.
I run this from cron every 5 minutes. As long as the redmine server and the server that the cron is run from both have accurate system clocks it should work well. A more elaborate version some day may include more error checking and dupe checking.
Have fun out there.
- Posted: 3 years ago
- Tags:code, ruby, twitter
I always have a dozen terminals open, so I figured it was time to make a way to post to twitter from the command line. This is extremely simple and can probably be improved, but for now this is working for me:
#!/usr/bin/env ruby
require 'rubygems'
require 'twitter'
if ARGV.size == 0
puts "\tUsage: tweet i like to eat cheese"
exit
end
#Load Configuration from ~/.twitter
begin
config = YAML::load(open(ENV['HOME'] + '/.twitter'))
rescue
puts "\tConfiguration File Missing"
puts "\tExample:\n\n"
puts "\tlogin: mylogin"
puts "\tpassword: mypassword"
puts "\n\tput in ~/.twitter\n"
exit
end
#initialize twitter library
twitter = Twitter::Base.new(config['login'], config['password'])
message = ARGV.join(' ').gsub(/"/, '')
status = twitter.post(message)
puts "\t\tTweet Posted: http://twitter.com/#{status.user.name}/statuses/#{status.id}\n"
Remember to 'gem install twitter' as root, or run as sudo, and also to put your configuration file in ~/.twitter. There is an example in the usage output of the script.
Update: I posted a new and slightly improved version of this for
Download here. This version will print out tweets for you if ran with no parameters, and will post any parameters you do use straight to twitter. Also, I found out that the twitter gem I am using already has a decent cli interface. Read about it
here.
- Posted: 3 years ago
- Tags:code, dspam, ruby
So finally, SpamAssasin just could not hack it any more on my server. This year there has been a steady influx of spam, and some users of my mail services are literally getting 1 good message out of 100 messages. I feel like I have truly exhausted all SA resources out there. 
From custom rule sets after more rule sets, after more configuration, SA just can't seem to learn fast enough. I also employ RBL/SBL checks and the works. So, what to do? Enter DSPAM.
Previously, I was a bit weary when I heard about the DSpam project and at the time, SpamAssassin was working for me, so why fix something that is not broken or so I thought. Well that mindset finally passed, and I dove into configuring DSpam for my setup. I run a Postfix/Cyrus type of a virtual email setup that is pretty complicated to say the least. I knew in advance that embarking down this road would mean learning some things and some elbow grease, but that is ok. I finally settled on a configuration that is similar to both Neale's Setup and Cepcep's Setup in different ways. In my setup, I have 2 independent chains, one for inbound email and one for outbound email. I decided to leave my outbound chain alone, and continue to send it thru amavisd. For the inbound chain, all mail is subject to getting sent to dspam, then dspam re-routes it back into postfix with a result attached.
Lastly, I wrote the following script, so that I may simply forward my mails to ham@ or spam@, and have postfix deliver it directly to my script, for retraining purposes.
It is piped to dspam, from postfix, like so:
dspam-retrain unix - n n - 10 pipe
flags=Ru user=dspam argv=ruby /usr/local/bin/dspam-retrain.rb $nexthop $sender $recipient
#!/usr/bin/ruby
# Ruby version of dspam-retrain perl script.
# Perl version: http://dspamwiki.expass.de/DspamRetrainScript
# Author: wsr@rushforthnetworks.com
# License: BSD
# Abstract: setup postfix to pipe mail into this script
# so that we may then pass it off to dspam in
# an appropriate way.
#
# dspam-retrain.rb handles spam-user@domain.tld, and
# spam@domain.tld formats
#
# If dspam-retrain.rb does not find a signature,
# it will train dspam using corpus or innoculation
# mode.
#
# Requirements: open4 gem is installed
# -----------------------------------------------------------
#### Configuration BEGIN ####################################
#enable logging?
@enable_logging = true
@log_file = '/tmp/dspam_retrain.log'
#what mode to train dspam in if we do not find signature?
#corpus or innoculation
@alternative_mode = 'corpus'
#### Configuration END #####################################
require 'rubygems'
require 'open4'
if @enable_logging
require 'logger'
@logfile = File.new(@log_file, 'a+')
@log = Logger.new(@logfile)
end
def logthis(message)
if @enable_logging
@log.info(message)
end
end
# Get arguments
spam_class = ARGV[0]
sender = ARGV[1]
recip = ARGV[2]
logthis("dspam-retrain Started. Arguments: #{spam_class}, #{sender}, #{recip}")
#see if we were passed spam-user@ or just user@
if match = recip.to_s.match(/^(spam|ham)-(\w+)@/)
user = recip.gsub(/#{match[1]}\-/, '')
elsif match = recip.to_s.match(/^(\w+)@/)
user = sender
else
logthis("\tCant't determine user")
exit 75
end
signature = String.new
message = String.new
#loop through email (passed via stdinput)
#search for signature
$stdin.each do |line|
if line.match(/X-DSPAM-Signature/)
signature = line.gsub(/X-DSPAM-Signature:/, '')
#remove any potential whitespace
signature.strip!
#since we found signature, break loop
break
end
message << line
end
if signature.length.to_i == 0
#we did not find a signature, do normal training
mode = 'train'
logthis("\tEmail did not have signature passed in. Attempting #{@alternative_mode} train.")
#open up dspam with appropriate options
pid, dspam_in, dspam_out, dspam_err = Open4::popen4 "/usr/bin/dspam --source=#{@alternative_mode} --class=#{spam_class} --user #{user}"
#attempt to feed message in
begin
dspam_in << message
rescue
#means dspam closed stdinput because our
#options failed
dspam_err.each_line do |o|
logthis("\t" + o.gsub(/\n/, ''))
end
end
#close dspams stdinput
dspam_in.close_write
#see if dspam left any messages for us
dspam_out.each_line do |o|
if o.strip.length == 0 then next end
logthis("\t" + o.gsub(/\n/, ''))
end
else
#we found signature, so we will only pass that.
mode = 'retrain'
logthis("\tRetraining Signature: #{signature} for User: #{user} as: #{spam_class}")
#open up dspam with appropriate options
pid, dspam_in, dspam_out, dspam_err = Open4::popen4 "/usr/bin/dspam --source=error --signature=#{signature} --class=#{spam_class} --user #{user}"
#see if dspam left any messages for us
dspam_out.each_line do |o|
if o.strip.length == 0 then next end
logthis("\t" + o.gsub(/\n/,''))
end
dspam_err.each_line do |o|
if o.strip.length == 0 then next end
logthis("\t" + o.gsub(/\n/,''))
end
end
ignored, status = Process::waitpid2 pid
#see if we exited cleanly and log it
if status.exitstatus == 0
logthis("\tMessage successfully #{mode}ed as #{spam_class}")
else
logthis("\tMessage NOT #{mode}ed")
end
Older Entries »
