remember when the links out of the rte looked like http://typo3/http://yourlink.tld/ ? well, as i still prod at links looking like this, i’d rather post the relieving sql statement here. it’s simple but handy. the replace function just makes backend work even faster.
UPDATE tt_content SET bodytext = REPLACE(bodytext, 'http://typo3/http://', 'http://');
Little problem after update to TYPo3 4.3 alpha3 i.e. with domain entries. After changing the domain record for a site (in a multi domain tree setup) the following error is displayed:
Simple solution: go to tools>install enter the password and start the database analyser and let the compare module do it´s job. after that, most problems like the above should be gone. consecutive action might be the db check>Check and update global reference index.
For most, it is obvious that the compare module is the successional move to finish the incremental update. But if you forget, you might find that blog post. things done ^^
a nice tool to go frenzy on a webserver is siege. it simulates an editable amount of concurrent users that pick on the web server for a variable time. having found this tool, stress testing some TYPO3 instances on apache(s) i rule was the inevidable next step.
now first of all this is not my daily business, but it’s tending to become to, so if you have corrections, suggestions and comments, your lines are appreciated below. the post you read is more like a small tutorial for those who want to know how performant the own webserver (i.e. apache) does. i like siege because of the easy settings, but i also run the better known apache benchmark when it comes down to a single page.
get all urls from the system, it’s willing to accept and put them in a text file line by line. my TYPO3 (4.3.0a3) runs realurl, so the trick is easy.
SELECT spurl FROM `tx_realurl_urldecodecache` WHERE 1=1 LIMIT 0 , 4500
sure, one could narrow the links down by using WHERE rootpage_id = [domain1_id] but as the whole server is to be tested, give me what i can get.
Clear cache for all domains in tree
Here you can decide what to do when clearing the frontend cache. By default all static html files will be deleted. Usually this is fine. Most installations of TYPO3 serve a single domain. If multiple domains are served from the same TYPO3 tree you might want to leave the cache for the other domains intact. If you uncheck the ‘clearCacheForAllDomains’ checkbox, only the html files are removed that are in the same domain as which you are logged into the backend.
me@machine1 [/home/me] siege -c 60 -t 5M -i -b -f realurls.txt > /dev/null
-c 60 # makes siege simulate 60 concurrent web users, i think that’s a number
-t 5M # the battle will last around 600 seconds
-b # benchmarking mode, leave no delay between requests -> make the box steam!
-i -f realurls.txt # internet activity mode, the links in realurls.txt are randomly clicked, rather than sequentially
> /dev/null # the output is going to the built-in black hole, better than flaming my console. you may prefer logging to a file.csv which contains info on the single requests (useful for finding the slow pages)
i was prepared for stuff but that hit me down (hopefully your first values are better)… after a while results are shown.
| Response time: | 4.15 secs |
|---|---|
| Transaction rate: | 12.06 trans/sec |
| Throughput: | 0.28 MB/sec |
wtf! the machine got spikes in the load like 25! this is not what i expected. i was aiming for about a mean response time of shorter than a second and, #whatdoiknow, about a hundred successfull transactions per second?
now it’s definitely time to optimize some settings. right now (two days later), the same box drives like
| Response time: | 0.13 secs |
|---|---|
| Transaction rate: | 507.40 trans/sec |
| Throughput: | 2.78 MB/sec |
load below 4. not bad, hu? most tweaks (apache, mysql, TYPO3) i did to achieve this will be written in one of the next posts, as i’m not finished yet ^ #botmustwork
(you see result of iteration 1 and i think 8. the siege settings haven’t changed, of course cache revalidation was disabled)
if you’ve got a web site up and running, you’d better be looking on performance in the real world. is the page digg-proof and does it stand a large amount of requests over time, even when cache was cleared? siege can help in doing a forecast.
after the site gone public, you will try to avoid becoming host of dead ends.
| 0.27 MB/sec |
one step closer to TYPO3 backend convenience
scheduled update of the extension list from the TYPO3 extension repository (TER). the extension needed has the cryptic key rscliem and will provide command line interface function for extension manager operations. just import and install it.
If you have no BE-user named _cli_lowlevel go and create it. That one’s needed to run PHP CLI commands.
/var/www/my-t3site/typo3/cli_dispatch.phpsh emtools --update-extensionlist
It will return a fancy array containing a message like
[0] => <p>The extension list has not changed remotely, it has thus not been fetched.</p>
(at least if you run the command successfully a second time)
To have the Extension List updated twice a day, a small entry in the current cron tab comes in handy:
# m h dom mon dow command 00 */12 * * * /var/www/my-t3site/typo3/cli_dispatch.phpsh emtools --update-extensionlist MAILTO="adminmail@mydomain.com"
The following options can be used:
- –update-mirrors
Updates mirror list from typo3.org- –update-extensionlist
Retrieves current extensionlist from selected mirror and load into database- –check-extensions
Checks currently installed extension for newer versions- –import-updates
Imports updates for all extension, found with –check-extensions.
(taken from the README file found in typo3conf/ext/rscliem/)
You see, you can possibly automate the extension update process. I currently can’t use this feature because some extensions would break the TYPO3 instance in question if updated to newest version.
But the handy part is to have the list of extensions to update emailed to me.A connection of the updated extension list and the extension chack makes most sense.
./cli_dispatch.phpsh emtools --update-extensionlist && ./cli_dispatch.phpsh emtools --check-extensions
small snippet great effect to your typoscript. i use this to spare lines of executed typoscript if the editor won’t use it anyways. in my opinion it’s not only a great tool for extension development, but also it can restructure parts of your ts setup of the site.
constants:
SHOWBADGE = 1
setup:
# only work out stuff for SHOWBADGE is true
[globalVar = LIT:1 = {$SHOWBADGE}]
page.10.marks.A < lib.showBadge
# etc...
[global]
i don’t know what the LIT:1 means, but it works. i mean, is there a LIT:2 and so on? if you’ve got crucial information on this matter, comment or mail me. thanks.
additional philosophy: what are constant conditions anyway?
if you’re about to work with the root page of some TYPO3 installation and, let’s say you have static_tables installed, you’re gonna get a long list of countries, markets and currencies in the root list which is fully loaded, everytime you want to administrate a backend user or a file mount – since these items are listed by default on the root page. i kinda got used to this.
but since this sucks a lot of time up in your backend session depending on server load and bandwidth, this one-liner found on the english TYPO3 mailing list can tweak your backend experience for the root page. the attempt of putting the line like “mod.web_list.static_template.hideTable = 1” into a pageTS will not work because the root page seems to have no pageTS at all.
instead, put this in the userTS of your be account or a user group you’re member of:
mod.web_list.hideTables=static_countries,static_country_zones,static_currencies,static_languages,static_territories,static_taxes,static_markets mod.web_list.hideTables=static_template
the tables you specify here will be gone on every page. in the case of static_* tables, they’re just to listed on the root page, anyway. so, listing your root page will speed up a lot!
here’s some useful snippet to control the size and the order of the access groups of a page.
standard size of the page access list
access user group listwhen working with larger TYPO3 sites, the access groups can become increasingly complex. the list view to select the groups from is about only five elements long by default, but that maybe a bit too short when it comes to more than 200 user groups out of i.e. an ldap structure.
this makes selecting the right group painful, especially when this list is not ordered by natural alphabetical order but by id or other fancy computer data.
// more space to select fe_groups $TCA['pages']['columns']['fe_group']['config']['size'] = 12; $TCA['pages']['columns']['fe_group']['config']['foreign_table_where'] = 'ORDER BY fe_groups.title'; $TCA['be_users']['columns']['usergroup']['config']['size'] = 12; $TCA['be_users']['columns']['usergroup']['config']['foreign_table_where'] = 'ORDER BY be_groups.title';
Add this code to your extTables.php file and things get smoother again. The last two lines apply the same for the backend user dialog with the backend user selection.
some of you may have noticed the Piwik link in my blog´s menu bar. well, I am using Piwik as open source website analysis tool for more projects now and it works just fine. this is why I just wanted to throw another blog post about it into the web. and here it is! Piwik is cool.
Other posts describe the pros and con of Piwik far better than I´d do in the next few minutes, that´s why I´ll tap it short:
For websites with some traffic, you´ll need quite more than the suggested minimum 128mb of memory for php. To have the Piwik interface sound and responding, you may want to precollect the tracking data with a cron job.
Generate a new user for Piwik ( piwik/index.php?module=UsersManager&action=index ) and copy the authorization token that you´ll get to paste it into the archive.sh file found under misc/cron/ in your Piwik folder.
crontab -u www-data -e
Append this line
5 0 * * * /path/to/piwik/misc/cron/archive.sh > /dev/null
Your box will respond way faster after having processed this. This is why I can check smoothly that it stais the way it used to be: my blog is being majorly visited with uptodate browsers and merely around 10% with IE or other funny stuff. Hoorray.