Squid3 Caching for xbox marketplace

Cheetohz

@Cheetohz:

@marcelloc:

check for pfsense disk performance optimizing. During install, you can enable soft updates. After install there are some sysctrl options to speed up disk.

I have 5x 75gb scsi drives in raid5, this "shouldn't" be a disk issue, however I have not looked into that possibility yet.

I monitored the initial download again after clearing the cache server. it was at ~15Mb from both WAN and Cache server. I think this may be an xbox limitation. I am checking on the xbox forum, though I doubt the professionals with the knowledge I am seeking are on that forum.

RAID5 Is the worst of em all as far as performance.

is it ~15Mb/s bad? i'd suspect better performance than that at least

marcelloc

The problem is not the disk itself. I had serious performance problems with pfsense and postfix package on ibm 3650. After enabling soft updates I coud get all disk performance.

Try scp a file from and to pfsense to check how fast you can go.

periko

RAID5 Is the worst of em all as far as performance.

I agree, if u read a little about squid tunning, raid-5 is not on the list.

1; What are the technical details of your machine?
2; How much ram u have in your system?
3; use aufs, null is went u want to disable cache.
4; raid-1
5; U know the file size of the files u will be download? if can fit into your ram size for squid u can change maximum object size to that value if it fit into your ram size.
6; add noatime in the fstab file.
7; big files big nic 1GB+.
8; setting up 1TB for squid won't give u better results, start with 16GB, this will help the system to get the files more faster, the head of the disk will move less in 16GB than 100GB
I will setup the disk layout like this:

/ 8GB
/var 64GB

I will start the cache disk size wih 16GB and start caching and see access.log, run calamaris to see cache performance and start tuning the system base on 1 day caching.

add this to squid custom settings:

buffered_logs off
client_db off
log_fqdn off
log_icp_queries off
memory_pools off
check_hostnames off

my 2 cents.

Cheetohz

1. It's a dedicated Dell PowerEdge 2850, as you know running raid 5

2. has 4GB Ram

3. I am currently using aufs

4. I really don't want to destroy and rebuild the raid right now, but I will as a last resort.

5. The files will most likely exceed the total ram on the system.

6. I have doubts this will cause any significant increase in speed. I will try it and let you know the results.

7. Server has Gb nic, client has 100Mb, still we are only seeing about 1/10th of the proper speed. I also tested this on my PC with a 100MB file. I pulled it faster from the net than I did the cache server.

8. the total download is likely ~40GB, I will likely need at least 50GB cache to pull this off.

Version 2.1-RELEASE (amd64)
built on Wed Sep 11 18:17:37 EDT 2013
FreeBSD 8.3-RELEASE-p11

You are on the latest version.
Platform pfSense
CPU Type Intel(R) Xeon(TM) CPU 3.20GHz
4 CPUs: 2 package(s) x 1 core(s) x 2 HTT threads

State table size
0% (145/407000)

Show states
MBUF Usage
9% (2196/25600)

Load average
0.00, 0.00, 0.00

CPU usage
0%

Memory usage
17% of 4074 MB

SWAP usage
0% of 8192 MB

Disk usage
1% of 190G

I know, overkill for what I am using it for, but I don't see the system being the issue. I am barely touching its capabilities with pfsense.

Cheetohz

@marcelloc:

The problem is not the disk itself. I had serious performance problems with pfsense and postfix package on ibm 3650. After enabling soft updates I coud get all disk performance.

Try scp a file from and to pfsense to check how fast you can go.

SFTP will copy to and from at 35MB/s

Cheetohz

Just tested a download on my PC. 400-500KB from WAN, only 5-10KB from cache.

Also tested a local cp, 1GB file was completed in under 30 sec.

I think something else is wonky here.

periko

If u see the web page slow, try enable "resolv dns v4" check box, some times this cause issues.

Remember that the proxy will use first the RAM and will use the disk once he decide is time to send the object to hd.

Cheetohz

Everything pulling from cache is at 5KB/s now. I have to disable squid for now. Hopefully someone has more thoughts as to why this is happening.

serialdie

@Cheetohz:

Everything pulling from cache is at 5KB/s now. I have to disable squid for now. Hopefully someone has more thoughts as to why this is happening.

Have you try the stable build?
You are using the devel build.

Cheetohz

WOW! Problem solved!

Went to wipe the raid and install as raid 0

Found a failed drive and removed it. set up as raid 0 and did a fresh install, set everything back up manually. rebooted and it works like a champ.

Issue was I was running in degraded mode. I still find it interesting everything else was working like a champ.

periko

Great, if u are going to have this for a long time, move to raid1 for your own safety.
my 2 cents!!!

serialdie

@periko:

Great, if u are going to have this for a long time, move to raid1 for your own safety.
my 2 cents!!!

+1

Cheetohz

Now I get the Squid error page quite often, and certain sites seems to take longer to load.

It happens most frequently on these forums and on steam.

periko

If u see the web page slow, try enable "resolv dns v4" check box, some times this cause issues.

Is enable?

Cheetohz

@periko:

If u see the web page slow, try enable "resolv dns v4" check box, some times this cause issues.

Is enable?

This solved my issues, thank you very much!