Thursday, December 24, 2009

Java Native Memory Leak DeflaterOutputStream Derby or MySQL

So with some more recent database code. We we're compressing data elements before writing them to the database. Well our servers kept running out of memory. I mean Gigs and Gigs of memory.

Traced it down to if you don't call Session.End() on GZIPOutputStream or DeflaterOutputStream it will leak large amounts of Native memory.

This article has more details.

Native memory leaks suck to debug, took us about a week to find this. I hope this helps some other people.

Ultramon for Windows 7

Ultramon Beta is great for Windows 7

Works like a charm with even nice start menu;s

Download it here

Solution to Skype 3.8 on Windows 7

So I hate Skype 4.0 they just messed too much stuff up for me. So I like to run the old version of skype 3.8 but it doesn't work for Windows 7.

The solution, run Skype 3.6

I can confirm this build works (oldapps.com)

Alcohol 52% Windows 7 mount mds or iso Disk Images

Recently I upgraded to Windows 7. Alcohol 52% no longer works. :-(

Lucky Daemon Tools Lite works on Windows 7 and is a nice replacement. Sometimes I still use Alcohol 52% on Windows Vista to make images

Download this version

Saturday, July 11, 2009

C# POP3 Library with SSL for Gmail

So I ended up searching for hours trying to find a free C# library to do POP3 access to a SSL server like gmail. In our company we use Google Apps and we like gmail's mail server, it would be nice to be able to have programmatical access to these mail accounts. How does this not exist?

So this solution takes OpenPop.net Open source library and makes some very small tweaks so it has Ssl support.

Here is the important simple code snippet.
Stream actualstream = clientSocket.GetStream();

if (useSSL) {
System.Net.Security.SslStream sslstream = new SslStream(clientSocket.GetStream());
sslstream.AuthenticateAsClient(strHost);
actualstream = sslstream;
}


Attached here is the re zip of the source code with the change to the library and also a tweak to the sample app with a checkbox to include SSL support.

OpenPOP with SSL Source Download

OpenPOP with SSL Binaries Download

Enjoy!

Sunday, December 21, 2008

Generating Control Logic for 5 stage pipeline Processor Verilog

This one takes a little while to explain but is a really bright idea. Control logic for a processor is one of the most confusing parts. Especially if this is your first processor your building. Cranking out a Control Logic Table in excel or something is very valuable because it allows you to see how it all works out.

I wrote a little ruby script to generate verilog control code based on my spread sheet. Not only do you not have to code any control logic, but fixing control logic bugs are really easy because it is all in the spread sheet.

Ruby Script:

require 'pp'

def clean(str)
if str.rindex("b")
return str[1...str.size]
else
return str
end


end
@size = []
@wirename = []
@table = []

File.open('table.txt','r') do |f|
@size = f.gets.split("\t").map{|x| x.strip }
@wirename = f.gets.split("\t").map{|x| x.strip }
f.readlines.each do |l|
@table << l.split("\t").map{|x| x.strip }
end
end

@size.delete_at(0)
@size.delete_at(0)

@wirename.delete_at(0)
@wirename.delete_at(0)

@wiredefs = []
@assignments = []

#pp @size
#pp @wirename
#pp @table

#wire assignments
cur = 0
@wirename.each_with_index do |n,i|
puts n
df = ""
sz = @size[i].to_i
if sz == 1
df ="wire "
ass = "assign " + n.to_s + " = data[" + cur.to_s + "];"
else
df = "wire[" + (sz-1).to_s + ":0] "
ass = "assign " + n.to_s + " = data[" + (cur+sz-1).to_s + ":" + cur.to_s + "];"
end
df +=n.to_s + ";"


cur += sz
@wiredefs << df
@assignments << ass
end
datasize = 0
File.open ('export.v','w') do |v|

v.puts @wiredefs.join("\n")



@table.each {|x|
data = x[2...x.size]
data = data.reverse
data = data.map {|c| clean(c)}
q = data.join
code = "data = 32'b" + q
datasize = q.size if q.size > datasize


v.puts "7'b" + x[0] + ": " + code + " ;" + '//' + x[1].to_s
}

v.puts @assignments.join("\n")
end
puts "data size: " + datasize.to_s


Example Verilog output:

wire pc_en;
wire muxI;
wire[1:0] muxJ;
wire pc_select;
wire reg_write;
wire[1:0] rd_select;
wire sign_ext;
wire[1:0] sign_ext_mask;
wire[1:0] alu_rt_select;
wire alu_rs_select;
wire[1:0] alu_op;
wire[1:0] alu_cin_sign;
wire[1:0] alu_anot_bnot;
wire[1:0] shifter_op;
wire[2:0] logic_op;
wire exe_select;
wire save_select;
wire mem_write;
wire[2:0] write_select;
wire pc_add_select;
5'b00000: data = 32'bxxxx0xxxxxxxxxxxxxxxxxxxxx0xxxx0 ;//HALT
5'b00001: data = 32'bx0000x1xxxxx000100000xxxxx00xx01 ;//NOP
5'b01000: data = 32'bx0000x1xxxxx0001000101010010xx01 ;//ADDI
5'b01001: data = 32'bx0000x1xxxxx1011000101010010xx01 ;//SUBI
5'b01010: data = 32'bx0000x1xxxxx00xx100101000010xx01 ;//XORI
5'b01011: data = 32'bx0000x1xxxxx01xx110101000010xx01 ;//ANDNI
5'b10100: data = 32'bx0000x0xxx00xxxxxxx10xxx0010xx01 ;//ROLI
5'b10101: data = 32'bx0000x0xxx01xxxxxxx10xxx0010xx01 ;//SLLI
5'b10110: data = 32'bx0000x0xxx10xxxxxxx10xxx0010xx01 ;//SRAI
5'b10111: data = 32'bx0000x0xxx11xxxxxxx10xxx0010xx01 ;//SRLI
5'b10000: data = 32'bxxxx111xxxxx000100010101xx00xx01 ;//ST
5'b10001: data = 32'bx0100x1xxxxx0001000101010010xx01 ;//LD
5'b10011: data = 32'bx000111xxxxx0001000101010110xx01 ;//STU
5'b11001: data = 32'bx0010xx1xxxx00xx11001xxx1010xx01 ;//TM
5'b11011: data = 32'bx0000x1xxxxx000100001xxx1010xx01 ;//ADD
5'b11011: data = 32'bx0000x1xxxxx101100001xxx1010xx01 ;//SUB
5'b11011: data = 32'bx0000x1xxxxx00xx10001xxx1010xx01 ;//XOR
5'b11011: data = 32'bx0000x1xxxxx01xx11001xxx1010xx01 ;//ANDN
5'b11010: data = 32'bx0000x0xxx00xxxxxxx01xxx1010xx01 ;//ROL
5'b11010: data = 32'bx0000x0xxx01xxxxxxx01xxx1010xx01 ;//SLL
5'b11010: data = 32'bx0000x0xxx10xxxxxxx01xxx1010xx01 ;//SRA
5'b11010: data = 32'bx0000x0xxx11xxxxxxx01xxx1010xx01 ;//SRL
5'b11100: data = 32'bx0010xx000xx101000001xxx1010xx01 ;//SEQ
5'b11101: data = 32'bx0010xx001xx101000001xxx1010xx01 ;//SLT
5'b11110: data = 32'bx0010xx010xx101000001xxx1010xx01 ;//SLE
5'b11111: data = 32'bx0010xx011xx000000001xxx1010xx01 ;//SCO
5'b01100: data = 32'b0xxx0xxxxxxx00xx01000011xx0x0111 ;//BEQZ
5'b01101: data = 32'b0xxx0xxxxxxx00xx01000011xx0x0011 ;//BNEZ
5'b01110: data = 32'b0xxx0xxxxxxxxxxxxxxxx011xx0x1111 ;//BLTZ
5'b01111: data = 32'b0xxx0xxxxxxxxxxxxxxxx011xx0x1011 ;//BGEZ
5'b11000: data = 32'bx0110xxxxxxxxxxxxxx100110110xx01 ;//LBI
5'b10010: data = 32'bx0000x1xxxxx00xx011100100110xx01 ;//SLBI
5'b00100: data = 32'b0xxx0xxxxxxxxxxxxxxxx001xx01xx01 ;//J
5'b00101: data = 32'b1xxx0xxxxxxxxxxxxxxxx011xx01xx01 ;//JR
5'b00110: data = 32'b01000xxxxxxxxxxxxxxxx0011111xx01 ;//JAL
5'b00111: data = 32'b11000xxxxxxxxxxxxxxxx0111111xx01 ;//JALR
assign pc_en = data[0];
assign muxI = data[1];
assign muxJ = data[3:2];
assign pc_select = data[4];
assign reg_write = data[5];
assign rd_select = data[7:6];
assign sign_ext = data[8];
assign sign_ext_mask = data[10:9];
assign alu_rt_select = data[12:11];
assign alu_rs_select = data[13];
assign alu_op = data[15:14];
assign alu_cin_sign = data[17:16];
assign alu_anot_bnot = data[19:18];
assign shifter_op = data[21:20];
assign logic_op = data[24:22];
assign exe_select = data[25];
assign save_select = data[26];
assign mem_write = data[27];
assign write_select = data[30:28];
assign pc_add_select = data[31];


Example Excel File(csv):

1 1 2 1 1 2 1 2 2 1 2 2 2 2 3 1 1 1 3 1
pc_en muxI muxJ pc_select reg_write rd_select sign_ext sign_ext_mask alu_rt_select alu_rs_select alu_op alu_cin_sign alu_anot_bnot shifter_op logic_op exe_select save_select mem_write write_select pc_add_select
00000xx HALT 0 x bxx x 0 xx x xx xx x xx xx xx xx xxx x x 0 xxx x
00001xx NOP 1 0 bxx 0 0 bxx x xx b00 0 b00 b01 b00 bxx bxxx 1 x 0 b000 x
01000xx ADDI 1 0 bxx 0 1 b00 1 b10 b10 0 b00 b01 b00 bxx bxxx 1 x 0 b000 x
01001xx SUBI 1 0 bxx 0 1 b00 1 b10 b10 0 b00 b11 b10 bxx bxxx 1 x 0 b000 x
01010xx XORI 1 0 bxx 0 1 b00 0 b10 b10 0 b10 bxx b00 bxx bxxx 1 x 0 b000 x
01011xx ANDNI 1 0 bxx 0 1 b00 0 b10 b10 0 b11 bxx b01 bxx bxxx 1 x 0 b000 x
10100xx ROLI 1 0 bxx 0 1 b00 0 10 b10 x bxx bxx bxx b00 bxxx 0 x 0 b000 x
10101xx SLLI 1 0 bxx 0 1 b00 0 10 b10 x bxx bxx bxx b01 bxxx 0 x 0 b000 x
10110xx SRAI 1 0 bxx 0 1 b00 0 10 b10 x bxx bxx bxx b10 bxxx 0 x 0 b000 x
10111xx SRLI 1 0 bxx 0 1 b00 0 10 b10 x bxx bxx bxx b11 bxxx 0 x 0 b000 x
10000xx ST 1 0 bxx 0 0 bxx 1 b10 b10 0 b00 b01 b00 bxx bxxx 1 1 1 bxxx x
10001xx LD 1 0 bxx 0 1 b00 1 b10 b10 0 b00 b01 b00 bxx bxxx 1 x 0 b010 x
10011xx STU 1 0 bxx 0 1 b01 1 b10 b10 0 b00 b01 b00 bxx bxxx 1 1 1 b000 x
11001xx TM 1 0 bxx 0 1 b10 x bxx b01 0 b11 bxx b00 bxx b1xx x x 0 b001 x
1101100 ADD 1 0 bxx 0 1 b10 x bxx b01 0 b00 b01 b00 bxx bxxx 1 x 0 b000 x
1101101 SUB 1 0 bxx 0 1 b10 x bxx b01 0 b00 b11 b10 bxx bxxx 1 x 0 b000 x
1101110 XOR 1 0 bxx 0 1 b10 x bxx b01 0 b10 bxx b00 bxx bxxx 1 x 0 b000 x
1101111 ANDN 1 0 bxx 0 1 b10 x bxx b01 0 b11 bxx b01 bxx bxxx 1 x 0 b000 x
1101000 ROL 1 0 bxx 0 1 b10 x bxx b01 x bxx bxx bxx b00 bxxx 0 x 0 b000 x
1101001 SLL 1 0 bxx 0 1 b10 x bxx b01 x bxx bxx bxx b01 bxxx 0 x 0 b000 x
1101010 SRA 1 0 bxx 0 1 b10 x bxx b01 x bxx bxx bxx b10 bxxx 0 x 0 b000 x
1101011 SRL 1 0 bxx 0 1 b10 x bxx b01 x bxx bxx bxx b11 bxxx 0 x 0 b000 x
11100xx SEQ 1 0 bxx 0 1 b10 x bxx b01 0 b00 b10 b10 bxx b000 x x 0 b001 x
11101xx SLT 1 0 bxx 0 1 b10 x bxx b01 0 b00 b10 b10 bxx b001 x x 0 b001 x
11110xx SLE 1 0 bxx 0 1 b10 x bxx b01 0 b00 b10 b10 bxx b010 x x 0 b001 x
11111xx SCO 1 0 bxx 0 1 b10 x bxx b01 0 b00 b00 b00 bxx b011 x x 0 b001 x
01100xx BEQZ 1 1 b01 x 0 bxx 1 b01 b00 0 b01 bxx b00 bxx bxxx x x 0 bxxx 0
01101xx BNEZ 1 1 b00 x 0 bxx 1 b01 b00 0 b01 bxx b00 bxx bxxx x x 0 bxxx 0
01110xx BLTZ 1 1 b11 x 0 bxx 1 b01 bxx x bxx bxx bxx bxx bxxx x x 0 bxxx 0
01111xx BGEZ 1 1 b10 x 0 bxx 1 b01 bxx x bxx bxx bxx bxx bxxx x x 0 bxxx 0
11000xx LBI 1 0 bxx 0 1 b01 1 b01 b10 x bxx bxx bxx bxx bxxx x x 0 b011 x
10010xx SLBI 1 0 bxx 0 1 b01 0 b01 b10 1 b01 bxx b00 bxx bxxx 1 x 0 b000 x
00100xx J 1 0 bxx 1 0 bxx 1 b00 bxx x bxx bxx bxx bxx bxxx x x 0 bxxx 0
00101xx JR 1 0 bxx 1 0 bxx 1 b01 bxx x bxx bxx bxx bxx bxxx x x 0 bxxx 1
00110xx JAL 1 0 bxx 1 1 b11 1 b00 bxx x bxx bxx bxx bxx bxxx x x 0 b100 0
00111xx JALR 1 0 bxx 1 1 b11 1 b01 bxx x bxx bxx bxx bxx bxxx x x 0 b100 1

Lazy List in Ruby

A Lazy List is a list that evaluates and constructs it self as it is used. Very much a functional language concept.

class LazyList
attr_accessor :head, :tail


def initialize(h, t)
@head = h
@tail = t
end

def nil?
return head.nil?
end

def self.null
return LazyList.new(nil, nil)
end

def self.seq(start, finish)
if start > finish
return LazyList.null
else
return LazyList.new(start, proc { seq( start + 1, finish) })
end
end

def self.infseq(start)
return LazyList.new(start, proc { infseq( start + 1 ) })
end


def self.constList(val)
return LazyList.new(val, proc { constList(val) })
end

def self.boolseq(start)
return LazyList.new(start, proc { boolseq(!start) })
end


def self.filter(listcontrol, listdata)
if listcontrol.nil? || listdata.nil?
return LazyList.new
end

if listcontrol.head
LazyList.new(listdata.head, proc { filter(listcontrol.tail.call, listdata.tail.call)})
else
filter(listcontrol.tail.call, listdata.tail.call)
end

end

def Nth(n)
return if nil?

if n == 0
return head
else
tail.call.Nth(n-1)
end
end

def self.filter_check(data, check)
if nil?
return
end

if data.head % check != 0
LazyList.new(data.head, proc { filter_check(data.tail.call, check)})
else
filter_check(data.tail.call, check)
end

end

def self.prime_help(list)
return LazyList.new(list.head, proc { prime_help(filter_check(list.tail.call, list.head))})
end

def self.primes
return prime_help(infseq(2))
end

def printN(n)
if nil?
return
end

puts head
if n >= 0 && !tail.nil?
tail.call.printN(n-1)
end
end
end

l = LazyList.seq(2, 8)
l.printN(34)
l = LazyList.infseq(3)
l.printN(34)
b = LazyList.boolseq(false)
b.printN(6)
f = LazyList.filter(b,l)
f.printN(6)

a = LazyList.primes()
a.printN(8)
puts "nth #{f.Nth(2)}"