Benchmark Factory® for Databases
Version 5.5.1
Release Notes
February 2008
Quest Software is pleased to make available Benchmark Factory® for Databases 5.5.1 to meet your database workload testing needs. Benchmark Factory is a database performance and code scalability testing tool that simulates users executing transactions on the database and replays production workloads in non-production environments. This enables developers, DBAs, and QA teams to validate database scalability as their user load increases, application changes are made, and platform changes are implemented. Benchmark Factory is available for Oracle, SQL Server, DB2, Sybase, MySQL, and other databases via ODBC connectivity. Benchmark Factory:
Simulates thousands of concurrent users with a minimal amount of hardware.
Simulates database production workloads.
Reports tests results with built-in reporting features.
Upon completion of a test execution, all test results are collected and stored in the repository for data analysis and reporting. Benchmark Factory collects a variety of informative and detailed statistics that include:
Overall server throughput (measured in transactions per second/minute or bytes transferred).
Detailed transaction executed statistics by individual agent workstations producing a load.
Benchmark Factory Run Reports now allows you to copy general run and userload results to the Windows Clipboard and paste those results into the desired program.
The TPC-D Benchmark Scale can now be a value from 1 to 200,000,000 by 1. The scale factor of 0.1 is no longer valid.
Benchmark Factory is now integrated with Quest SQL Optimizer for Oracle, allowing you to test SQL statements for scalability under a specific user load to determine the best performing SQL alternatives.
Benchmark Factory now supports the Oracle 11g database.
The Oracle Trace File import process has been enhanced to allow for datetime bind parameters, NULL bind values, and better dependency filtering, as well as support Oracle 11g trace file import.
The SQL Server 7 trace file and SQL Server 2000/2005 workloads trace tables have been combined into single workload.
Note: Benchmark Factory no longer supports trace file import. To import a trace table, you must first save it as a trace table using SQL Optimizer, then import the table.
Benchmark Factory Run Reports now provides improved disclosure reporting of computed maximum qualified throughput at maximum user load for TPC-C tests and the power calculation and Timing Intervals for TPC-H tests.
An Agent Management tab has been added in Settings, allowing users to automatically start and shutdown agents.
The submit job process has been simplified to quickly run the desired test.
Benchmark Factory has some new advanced mapping options added for SQL Server and Sybase databases, allowing customization of the creation of benchmark objects.
A new setting has been added to limit the number of rows to be retrieved when executing custom SQL transactions.
Benchmark Factory can now reside on an Oracle database.
The following is a list of issues addressed in this release of Benchmark Factory for Databases.
|
Resolved Issue |
Change Request |
| Agent Column sorting not working for values under 1 second. | 38,238 |
| "No SQL to Test" error given when sending numerous SQLs from QCSYB SQL tuning to Benchmark Factory. | 38,320 |
| When running the TPC-C test with MS SQL Server ODBC driver errors generated. | 38,424 |
| Errors when creating DB2 tables using the tablespace mapping option. | 40,781 |
| Cannot import from PA SQL Server repository when datetime format in Benchmark Factory is English(UK). | 40,795 |
| The Realtime Summary Graph is not displaying properly in run reports. | 41,044 |
| From PA imports there is the need to display HOST-INSTANCE instead of just INSTANCE. | 43,742 |
| In Run Reports for the TPC-C disclosure report the average and max. response time values for Delivery and Stock Level are reversed (max <-> average). |
44,113
|
| tpmC values not calculated correctly for TPC-C test. |
44,114
|
| DB2 error when NULL is returned in result set for custom load scenario test. | 45,888 |
The following is a list of issues known to exist at the time of Benchmark Factory for Databases release.
|
Feature |
Known Issue |
Change Request |
| Benchmark Factory Console | Benchmark Factory does not support data types that are created by the user. | N/A |
| Trace file playback | When replaying very large trace file imports, for example, several thousand transactions, execution may take several minutes to start, or the execution may fail. | N/A |
If you have installed the Benchmark Factory for Databases Beta, uninstall first, before installing the latest version.
If you create a new Benchmark Factory 5.5.1 repository, earlier versions of Benchmark Factory will not work against this repository.
Client libraries for database types used during the load testing process must be installed on all testing machines (Benchmark Factory console and Agents).
The Benchmark Factory 3.3 files and below can be converted to Benchmark Factory 5.5.1 files. A project conversion program is available, and can be accessed by logging on to Quest Software, then select Downloads and Upgrades|Benchmark Factory for Databases|Special Patches| Benchmark Factory Project Converter.zip.
There is no upgrade path for the Benchmark Factory Repository version 3.3 or below.
Before installing Benchmark Factory for Databases 5.5.1, ensure your system meets the following minimum hardware and software requirements:
| Repository | Benchmark Factory cannot run without a repository, and by default
a MySQL repository is installed during the install process. After
installation, you can remove the MySQL service and delete the MySQL
repository, then select a repository of your choice. Benchmark Factory
supports the following databases:
|
| Platform | 1.0 GHz x86 compatible CPU |
| Memory | 512 MB of RAM recommended
Note: The memory required may vary based on the following:
|
| Hard Disk Space | 120 MB of disk space for installation
Note: The disk space required may vary based on options selected for installation. |
| Operating System | Microsoft®Windows® 2000, 2003, XP Professional, and Vista |
| Database Client/Sever Versions | Oracle: 8, 8i, 9i, 9.2, 10g, and 11g (Note:
Requires Oracle 8.1.7 client libraries or higher.) SQL Server: 7, 2000, and 2005 Sybase: 12.5 and 15 DB2 LUW: 8.1.5 through 9 MySQL: 4.5 and 5.0 |
This section contains information about installing and operating this product in non-English configurations, such as those needed by customers outside of North America. This section does not replace the materials about supported platforms and configurations found elsewhere in the product documentation.
The platforms and configurations supported in this release may use any time zone, character encoding, language, or locale configuration to operate this product. All components of this product must/should be configured to use the same or compatible character encodings and must/should be installed to use the same locale and/or regional options.
This release has been tested with the following language and platform combinations in addition to U.S. English:
The Benchmark Factory for Databases release package contains the following products:
| info@quest.com | |
| Quest Software, Inc. World Headquarters 5 Polaris Way Aliso Viejo, CA 92656 USA | |
| Web |
Refer to our Web site for regional and international office information.
Quest Support is available to customers who have a trial version of a Quest
product or who have purchased a commercial version and have a valid maintenance
contract.
Quest Support provides around the clock coverage with SupportLink,
our web self-service. Visit SupportLink at http://support.quest.com/.
From SupportLink, you can do the following:
View the Global Support Guide for a detailed explanation of support programs,
online services, contact information, and policy and procedures.
The guide
is available at: http://support.quest.com/pdfs/Global%20Support%20Guide.pdf.
This document contains proprietary information protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser's personal use without the written permission of Quest Software, Inc.
© 2008 Quest Software, Inc. ALL RIGHTS RESERVED.
Quest, Quest Software, the Quest Software logo, Aelita, AppAssure, Benchmark Factory, Big Brother, DataFactory, DeployDirector, ERDisk, Fastlane, Final, Foglight, Funnel Web, I/Watch, Imceda, InLook, IntelliProfile, Internet Weather Report, InTrust, IT Dad, JClass, Jint, JProbe, Knowledge Xpert, LeccoTech, LiteSpeed, LiveReorg, Matrix Insight, Matrix.Net, MIQ, NBSpool, NetBase, Npulse, PerformaSure, PL/Vision, Quest Central, RAPS, SharePlex, Sitraka, SmartAlarm, Speed Change Manager, Speed Coefficient, Spotlight, SQL Firewall, SQL Impact, SQL LiteSpeed, SQL Navigator, SQLab, SQLGuardian, SQLProtector, SQL Watch, Stat, Stat!, StealthCollect, Tag and Follow, Toad, T.O.A.D., Vintela, Virtual DBA, Xaffire, and XRT are trademarks and registered trademarks of Quest Software, Inc in the United States of America and other countries. Other trademarks and registered trademarks used in this guide are property of their respective owners.
If you have any questions regarding your potential use of this material, contact:
| Quest Software World Headquarters LEGAL Dept 5 Polaris Way Aliso Viejo, CA 92656 Email: legal@quest.com |
The information in this document is provided in connection with Quest products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest does not make any commitment to update the information contained in this document.