diff --git a/.gitignore b/.gitignore index 93a3523d..d2f6f674 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ *.class +dist/osgi/plugins/cosbench*.jar # Package Files # /0.3.0.10 diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 00000000..16f9e8a5 --- /dev/null +++ b/.travis.yml @@ -0,0 +1,9 @@ +language: java +java: + - openjdk6 + - oraclejdk6 + - openjdk7 + - oraclejdk7 + +install: PLEASE CHANGE ME +script: PLEASE CHANGE ME diff --git a/BUILD-openio.md b/BUILD-openio.md new file mode 100644 index 00000000..fe3bb913 --- /dev/null +++ b/BUILD-openio.md @@ -0,0 +1,36 @@ +Overview +-------- + +The following procedure indicates how to compile and package cosbench for OpenIO. +First, please read [BUILD.md](BUILD.md). + + +Development Environment +----------------------- + +To get a running comilation environment, please refer to the same section in [BUILD.md#development-environment](BUILD.md#development-environment). + +Note that Eclipse **4.7.3a** is a knwon working release. You must use **JDK 1.8** and configure eclipse this way: + +1. in `Window -> Preferences -> Java -> Prefered JRE` ensure a JDK 1.8 is selected. + +2. in `Window -> Preferences -> Java -> Compiler` ensure `Compiler compliance level` is set to **1.8**. + +Releasing new version +--------------------- + +When you are ready to release a new version: + +1. Update the file `VERSION`. + +2. Update the version in `dev/*/META-INF/MANIFEST.MF` by running the script `./version_unify.sh`. + +3. Remove previous JAR: `rm dist/osgi/plugins/cosbench-*.jar`. + +4. In eclipe, select all project, right click then `export ... -> Plug-in Development -> Deployable plugins and fragments`. + +5. Run `./pack-openio.sh` to generate `cosbench-$VERSION-openio.tar.gz`. + +6. Commit changes, tag to `${VERSION}-openio` and push to git (tagging can be done when making a new release from the github page). + +7. Create a release from [github page](https://github.com/open-io/cosbench/releases). diff --git a/BUILD.md b/BUILD.md index f93bf86c..7eac9843 100644 --- a/BUILD.md +++ b/BUILD.md @@ -30,7 +30,7 @@ Development Environment Below are steps to set up development environment in eclipse: -1. download eclipse SDK (Juno) from http://www.eclipse.org/downloads/ +1. download eclipse SDK from [http://download.eclipse.org/eclipse/downloads/](http://download.eclipse.org/eclipse/downloads/), and we are using the version 4.2.1 (Juno). 2. get cosbench source code tree by git or downloading the whole zip package. diff --git a/CHANGELOG b/CHANGELOG index 19652d35..282b090d 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,79 +1,108 @@ + COSBench 0.4.2.c3(15-ww32.5) +------------------------------------ +- #279: "IsAlive" for drivers goes red after a while +- #278: unable to run workload against HP public cloud +- #276: no response with any data when running on RHEL 7.1 +- #275: It appears that Cosbench does not properly close driver sockets. +- #274: FreeMarker template error: The following has evaluated to null or missing ... +- #270: S3: Timeout modifies only the connection timeout + + + COSBench 0.4.2.c2(15-ww15.3) +------------------------------------ +- #267: v0.4.2.c1: Swift/Keystone regression +- #265: S3: Need configurable max_connections for AmazonS3Client +- #260: when testing ceph giant through librados adapter, some errors like EINPROGRESS raises +- #254: Mixed Workloads (Read,Write,Delete) cause low read success rate. +- #227: CLI script doesn't show current stage +- #87: [WorkloadConfigurator] Add support for specifying number of drivers and workers-per-driver (extension to #86) +- #5: real-time performance grap +- #221: Dumping of Per driver result +- #210: multiple drivers on the same physical node +- #252: too many open files” errors with filewrite type +- #236: Keystone needs to accept region name as parameter. +- #250: hard to distinguish different operations under the same work item from U +- #238: Working with Ceph 0.87 encountered errors +- #245: Driver not alive +- #240: cosbench is not starting under CentOS + + COSBench 0.4.1.0(14-ww53.2) ------------------------------------ -. Add one workload under workloads folder, whose distribution is summarized from two week's trace on ASUS WebStorage. Thanks ASUS' contribution! -. #223: current authorization doesn't support retry -. #222: completed operation count is mismatching with configured totalOps -. #220: UI: The current stage table shouldn't merge with stage table when workload terminates -. #212 swift show 401 error code -. #194 Config UI: Add detail to General Report Table located in view details section of Active Workloads : Need to add information about stage names -. #188 UI: Don't generate stagenames with comma characters in them +- Add one workload under workloads folder, whose distribution is summarized from two week's trace on ASUS WebStorage. Thanks ASUS' contribution! +- #223: current authorization doesn't support retry +- #222: completed operation count is mismatching with configured totalOps +- #220: UI: The current stage table shouldn't merge with stage table when workload terminates +- #212 swift show 401 error code +- #194 Config UI: Add detail to General Report Table located in view details section of Active Workloads : Need to add information about stage names +- #188 UI: Don't generate stagenames with comma characters in them COSBench 0.4.0.1(14-ww48.5) ------------------------------------ -. #214: ceph prepare stage very low success ratio and throwing erros -. #211: SwiftTokenCache needs to validate current token is valid -. #203: the driver "isAlive" status is green even the driver is not started -. #198: in v0.4.0.0, it seems the AFR doesn't take in effect in prepare stage -. #191: cosbench-start.sh fails while executing 'nc': TOOL_PARAMS is incorrect. -. #190: Config UI: Add detail to cleanup/dispose stage names -. #186: UI: Add detail to init and prepare stage names -. #178: Collect and view results for user-terminated jobs -. #159: Extend "histogram" selector to support "floating point" weights/percentages -. #156: Add "trigger" support -. #154: Add authentication caching support -. #147: UI: Advanced config UI Cleanup (part 1) -. #146: UI: Add tooltips to configurable params on the config UI -. #145: UI: Change storage url template depending on the storage type chosen -. #32: digest authentication support -. #29: Error handling improvement -. #19: Support LIST Api -. #12: after several hours running, cosbench's banners all disappear. +- #214: ceph prepare stage very low success ratio and throwing erros +- #211: SwiftTokenCache needs to validate current token is valid +- #203: the driver "isAlive" status is green even the driver is not started +- #198: in v0.4.0.0, it seems the AFR doesn't take in effect in prepare stage +- #191: cosbench-start.sh fails while executing 'nc': TOOL_PARAMS is incorrect. +- #190: Config UI: Add detail to cleanup/dispose stage names +- #186: UI: Add detail to init and prepare stage names +- #178: Collect and view results for user-terminated jobs +- #159: Extend "histogram" selector to support "floating point" weights/percentages +- #156: Add "trigger" support +- #154: Add authentication caching support +- #147: UI: Advanced config UI Cleanup (part 1) +- #146: UI: Add tooltips to configurable params on the config UI +- #145: UI: Change storage url template depending on the storage type chosen +- #32: digest authentication support +- #29: Error handling improvement +- #19: Support LIST Api +- #12: after several hours running, cosbench's banners all disappear. COSBench 0.4.0.0 (14-ww21.5) ------------------------------------ -. #57: time synchronization -. #96: Creating Web UI facility for generating different workload configuration files -. #155: Add support for "acceptable failure rate, failure %" to all stages -. #4: xml validator -. #77: contextual parameter list -. #167: storage policy supporting for openstack swift -. #37: support CDMI interface -. #139: NullPointerException when a mission is aborted -. #166: show driver alive state on controller index page -. #158: Extend "histogram" selector to support "open ranges" -. #165: enable Open range for object size in uniform and histogram -. #136: 1 GB Object Reads Fail due to IllegalArgumentException -. #99 (#152): measure and report the time for first and last byte in or out -. #159: Extend "histogram" selector to support "floating point" weights/percentages -. #143: at loading archived workloads, even there are 100 workloads, the UI shows only 98. -. #140: duplicated workload id generated -. #151: avoid being challenged for each request -. #137: when there are large # of archive jobs, exception will raise from archive list on web portal. -. #139: NullPointerException when a mission is aborted -. support http BASIC and DIGEST authentication -. add cdmi base and util bundle -. add cdmi adapter for swift+cdmi middleware +- #57: time synchronization +- #96: Creating Web UI facility for generating different workload configuration files +- #155: Add support for "acceptable failure rate, failure %" to all stages +- #4: xml validator +- #77: contextual parameter list +- #167: storage policy supporting for openstack swift +- #37: support CDMI interface +- #139: NullPointerException when a mission is aborted +- #166: show driver alive state on controller index page +- #158: Extend "histogram" selector to support "open ranges" +- #165: enable Open range for object size in uniform and histogram +- #136: 1 GB Object Reads Fail due to IllegalArgumentException +- #99 (#152): measure and report the time for first and last byte in or out +- #159: Extend "histogram" selector to support "floating point" weights/percentages +- #143: at loading archived workloads, even there are 100 workloads, the UI shows only 98. +- #140: duplicated workload id generated +- #151: avoid being challenged for each request +- #137: when there are large # of archive jobs, exception will raise from archive list on web portal. +- #139: NullPointerException when a mission is aborted +- support http BASIC and DIGEST authentication +- add cdmi base and util bundle +- add cdmi adapter for swift+cdmi middleware COSBench 0.3.3.0 (13-ww44.5) ------------------------------------ -. #20: multiple same operations -. #16: web console authentication -. #34: 0% success on write -. #11: job management -. #70: multiple main stages -. #30: no historical workloads after restart -. add histogram selector to define histogram size distribution -. #58: sleep between stage -. #84: multiple init/prepare/cleanup/dispose from config UI -. oen bug fix for java.lang.ArrayIndexOutOfBoundsException at data archiving -. #118: file-write operation - percentile zero issue -. #121: specify archive dir -. #92: using swift storage without authentication -. support auth token caching for swift testing -. #128: Use workstage name from XML rather than generic names in logs +- #20: multiple same operations +- #16: web console authentication +- #34: 0% success on write +- #11: job management +- #70: multiple main stages +- #30: no historical workloads after restart +- add histogram selector to define histogram size distribution +- #58: sleep between stage +- #84: multiple init/prepare/cleanup/dispose from config UI +- oen bug fix for java.lang.ArrayIndexOutOfBoundsException at data archiving +- #118: file-write operation - percentile zero issue +- #121: specify archive dir +- #92: using swift storage without authentication +- support auth token caching for swift testing +- #128: Use workstage name from XML rather than generic names in logs COSBench 0.3.1.2 (13-ww31.3) @@ -115,4 +144,4 @@ COSBench 0.3.0.1 (13-ww17.5) COSBench 0.3.0.0 (13-ww15.5) ------------------------------------ -1. rebase version to 0.3.0 for open source \ No newline at end of file +1. rebase version to 0.3.0 for open source diff --git a/COSBench-Quick-Start.pdf b/COSBench-Quick-Start.pdf new file mode 100644 index 00000000..75d12ef7 Binary files /dev/null and b/COSBench-Quick-Start.pdf differ diff --git a/COSBench-User-Guide.odt b/COSBench-User-Guide.odt index 2457ec0c..594b7a28 100644 Binary files a/COSBench-User-Guide.odt and b/COSBench-User-Guide.odt differ diff --git a/COSBenchUserGuide.pdf b/COSBenchUserGuide.pdf index 139087f1..ca2690cf 100644 Binary files a/COSBenchUserGuide.pdf and b/COSBenchUserGuide.pdf differ diff --git a/LICENSE b/LICENSE index 62610ec0..dfbe79ed 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,5 @@ Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -10,4 +11,4 @@ Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. \ No newline at end of file +limitations under the License. diff --git a/README.md b/README.md index 60e9951e..0ccf840b 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ COSBench is a benchmarking tool to measure the performance of Cloud Object Stora emerging technology that is different from traditional file systems (e.g., NFS) or block device systems (e.g., iSCSI). Amazon S3 and Openstack* swift are well-known object storage solutions. -COSBench now supports OpenStack* Swift, Amazon* S3, Amplidata v2.3, 2.5 and 3.1, Scality*, Ceph, CDMI, Google* Cloud Storage, Aliyun OSS as well as custom adaptors. +COSBench now supports OpenStack* Swift, Amazon* S3, OpenIO*, Amplidata v2.3, 2.5 and 3.1, Scality*, Ceph, CDMI, Google* Cloud Storage, Aliyun OSS as well as custom adaptors. Important Notice and Contact Information diff --git a/TODO.md b/TODO.md index e8eae2c2..f71ecaab 100644 --- a/TODO.md +++ b/TODO.md @@ -4,14 +4,8 @@ COSBench is an ongoing project, in next six months, we plan to make below improv 1. more storage interface support, list is TBD. -2. authentication caching - -3. LIST operation - -4. workload packages shipped - If you have any other suggestions or you want to work with us together. please contact us at yaguang.wang@intel.com. -== END == \ No newline at end of file +== END == diff --git a/VERSION b/VERSION index 2b7c5ae0..f9056827 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.4.2 +0.4.7 diff --git a/datescript.vbs b/datescript.vbs new file mode 100644 index 00000000..1e99176f --- /dev/null +++ b/datescript.vbs @@ -0,0 +1 @@ +wscript.echo year(date) & right( "0" & month(date),2) & right( "0" & day(date),2) diff --git a/dev/cosbench-ampli/META-INF/MANIFEST.MF b/dev/cosbench-ampli/META-INF/MANIFEST.MF index b2010dc2..8c21ec72 100644 --- a/dev/cosbench-ampli/META-INF/MANIFEST.MF +++ b/dev/cosbench-ampli/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Cosbench Ampli Client Bundle Bundle-SymbolicName: cosbench-ampli -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.context, diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/api/amplistor/AmpliStorage.java b/dev/cosbench-ampli/src/com/intel/cosbench/api/amplistor/AmpliStorage.java index 0ec4433e..c7d2dfa4 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/api/amplistor/AmpliStorage.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/api/amplistor/AmpliStorage.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.amplistor; @@ -34,9 +34,9 @@ /** * The AmpliStor implementation for Storage API. - * + * * @author ywang19, qzheng7 - * + * */ public class AmpliStorage extends NoneStorage { @@ -70,7 +70,7 @@ public void init(Config config, Logger logger) { parms.put(TIMEOUT_KEY, timeout); logger.debug("using storage config: {}", parms); - + HttpClient httpClient = HttpClientUtil.createHttpClient(timeout); client = new AmpliClient(httpClient, host, port, ns_root); logger.debug("ampli client has been initialized"); @@ -89,10 +89,10 @@ public void dispose() { @Override public void abort() { - super.abort(); - client.abort(); + super.abort(); + client.abort(); } - + @Override public InputStream getObject(String container, String object, Config config) { super.getObject(container, object, config); diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/api/amplistor/AmpliStorageFactory.java b/dev/cosbench-ampli/src/com/intel/cosbench/api/amplistor/AmpliStorageFactory.java index 5cc1a841..797d9390 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/api/amplistor/AmpliStorageFactory.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/api/amplistor/AmpliStorageFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.amplistor; diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliClient.java b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliClient.java index 89ae0f28..be7b31c0 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliClient.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliClient.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.amplistor; @@ -33,9 +33,9 @@ /** * This class encapsulates AmpliStor related REST operations, so far no * authentication supported. - * + * * @author ywang19, qzheng7 - * + * */ public class AmpliClient { @@ -54,7 +54,7 @@ public AmpliClient(HttpClient client, String host, int port, String nsRoot) { } public void dispose() { - request = null; + request = null; HttpClientUtil.disposeHttpClient(client); } @@ -63,7 +63,7 @@ public void abort() { request.abort(); request = null; } - + public boolean login() throws IOException, HttpException { String storageUrl = "http://" + this.host + ":" + this.port; @@ -183,7 +183,7 @@ public String StoreStreamedObject(InputStream stream, long length, * unauthorized, 2. server say 401 Unauthorized, use Digest, 3. client * retries with Digest, but this will fail, due to the nature of * non-repeatable streamed entity). - * + * * the workaround is to convert streamed (non-repeatable) entity to * self-contained (repeatable). */ @@ -197,7 +197,7 @@ public String StoreObject(byte[] data, String ampliNamespace, + HttpClientUtil.encodeURL(ampliFilename)); method.setHeader("Content-Type", "application/octet-stream"); - + HttpResponse response = null; try { method.setEntity(new ByteArrayEntity(data)); @@ -253,12 +253,12 @@ public InputStream getObjectAsStream(String namespace, String objName) + "/" + HttpClientUtil.encodeURL(objName)); HttpResponse response = null; - - response = client.execute(method); + + response = client.execute(method); if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) { return response.getEntity().getContent(); - } + } EntityUtils.consume(response.getEntity()); if (response.getStatusLine().getStatusCode() == HttpStatus.SC_NOT_FOUND) { return response.getEntity().getContent(); @@ -274,7 +274,7 @@ public boolean deleteObject(String ampliNamespace, String name) HttpDelete method = null; HttpResponse response = null; - + try { String storageUrl = "http://" + this.host + ":" + this.port + nsRoot; @@ -531,24 +531,24 @@ public Map getObjectMetadata(String namespace, HttpResponse response = null; try { - - response = client.execute(method); - - if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) { - Header[] headers = response.getAllHeaders(); - Map map = new HashMap(); - - for (Header header : headers) { - map.put(header.getName(), header.getValue()); - } - - return map; - } else { - throw new AmpliException( - "unexpected error when request object metadata " + objName - + "@" + namespace, response.getAllHeaders(), - response.getStatusLine()); - } + + response = client.execute(method); + + if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) { + Header[] headers = response.getAllHeaders(); + Map map = new HashMap(); + + for (Header header : headers) { + map.put(header.getName(), header.getValue()); + } + + return map; + } else { + throw new AmpliException( + "unexpected error when request object metadata " + objName + + "@" + namespace, response.getAllHeaders(), + response.getStatusLine()); + } } finally { if (response != null) EntityUtils.consume(response.getEntity()); diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliConstants.java b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliConstants.java index ca1005f0..d89e97c0 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliConstants.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliConstants.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.amplistor; @@ -25,18 +25,18 @@ public interface AmpliConstants { String TIMEOUT_KEY = "timeout"; Integer TIMEOUT_DEFAULT = 30000; - + String HOST_KEY = "host"; String HOST_DEFAULT = "127.0.0.1"; - + String PORT_KEY = "port"; int PORT_DEFAULT = 8080; - + String NSROOT_KEY = "nsroot"; String NSROOT_DEFAULT = "/namespace"; - + String POLICY_KEY = "policy"; String POLICY_DEFAULT = ""; - + } diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliException.java b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliException.java index 841dd712..5e4961fd 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliException.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.amplistor; @@ -21,7 +21,7 @@ /** * @author ywang19 - * + * */ public class AmpliException extends Exception { diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliNamespace.java b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliNamespace.java index c8c5d6ac..c16eec71 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliNamespace.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliNamespace.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.amplistor; diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliNotFoundException.java b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliNotFoundException.java index f3a0dfac..5e4f37ef 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliNotFoundException.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliNotFoundException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.amplistor; @@ -23,8 +23,8 @@ public class AmpliNotFoundException extends AmpliException { /** - * - */ + * + */ private static final long serialVersionUID = -8231309074378291367L; public AmpliNotFoundException(String message, Header[] httpHeaders, diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliObject.java b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliObject.java index b12de6e5..f8a3f148 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliObject.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliObject.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.amplistor; diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliPolicy.java b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliPolicy.java index 8fa6c90d..e2c31398 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliPolicy.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliPolicy.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.amplistor; diff --git a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliUtils.java b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliUtils.java index 4b528c3d..10f53b93 100644 --- a/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliUtils.java +++ b/dev/cosbench-ampli/src/com/intel/cosbench/client/amplistor/AmpliUtils.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.amplistor; diff --git a/dev/cosbench-api/META-INF/MANIFEST.MF b/dev/cosbench-api/META-INF/MANIFEST.MF index eb1091ba..e1b4ddcf 100644 --- a/dev/cosbench-api/META-INF/MANIFEST.MF +++ b/dev/cosbench-api/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench API Bundle Bundle-SymbolicName: cosbench-api -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.config, diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPI.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPI.java index d7228c1e..f92863d7 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPI.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPI.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; @@ -27,35 +27,35 @@ public interface AuthAPI { * Initializes an Auth-API with parameters contained in the * given config, whose content depends on the specific Auth * type. Normally, it will also initialize one client for authentication. - * + * * @param config * - one instance from com.intel.cosbench.config.Config, which * includes parameters for authentication, and it will be passed * from execution engine. * @param logger * - one instance from com.intel.cosbench.log.Logger, which - * delivers logging capabilities to Auth-API, and it will be passed + * delivers logging capabilities to Auth-API, and it will be passed * from execution engine. */ public void init(Config config, Logger logger); - + /** - * Initializes a new HttpClient for WorkAgent to relogin. + * Initializes a new HttpClient for WorkAgent to relogin. */ public void init(); /** * Triggers the back-end authentication mechanism. - * + * * @return AuthContext - one com.intel.cosbench.context.AuthContext instance which contains all parameters - * configured for the authentication mechanism if authentication is successful, and otherwise, an + * configured for the authentication mechanism if authentication is successful, and otherwise, an * exception will be raised. */ public AuthContext login(); /** * Retrieves parameters and current settings used by the Auth-API. - * + * * @return Context - one com.intel.cosbenchc.context.Context instance which contains all parameters * configured for the authentication mechanism. */ diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPIFactory.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPIFactory.java index 6aa19332..d0354864 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPIFactory.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPIFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPIService.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPIService.java index e0d27436..fd8764a4 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPIService.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthAPIService.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthBadException.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthBadException.java index 4fe7c115..2367ab6d 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthBadException.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthBadException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthConstants.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthConstants.java index ff5210fb..a33ff853 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthConstants.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthConstants.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; @@ -21,8 +21,8 @@ public interface AuthConstants { String LOGGING_KEY = "logging"; Boolean LOGGING_DEFAULT = Boolean.FALSE; - + String CACHING_KEY = "caching"; - Boolean CACHING_DEFAULT = Boolean.FALSE; + Boolean CACHING_DEFAULT = Boolean.FALSE; } diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthException.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthException.java index f97dfff8..b7fea5f2 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthException.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthInterruptedException.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthInterruptedException.java index 9f5aaef7..3e3ed1f2 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthInterruptedException.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthInterruptedException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthTimeoutException.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthTimeoutException.java index 84df6ce0..c8b3d911 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthTimeoutException.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/AuthTimeoutException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/auth/NoneAuth.java b/dev/cosbench-api/src/com/intel/cosbench/api/auth/NoneAuth.java index 71aa13a4..494ad71e 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/auth/NoneAuth.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/auth/NoneAuth.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.auth; @@ -26,9 +26,9 @@ /** * This class encapsulates one none authentication mechanism which is used if no * any other authentication mechanism is assigned. - * + * * @author ywang19, qzheng7 - * + * */ public class NoneAuth implements AuthAPI { @@ -49,19 +49,19 @@ public NoneAuth() { public void init(Config config, Logger logger) { this.logger = logger; this.parms = new DefaultAuthContext(); - + logging = config.getBoolean(LOGGING_KEY, LOGGING_DEFAULT); caching = config.getBoolean(CACHING_KEY, CACHING_DEFAULT); - + /* register all parameters */ parms.put(LOGGING_KEY, logging); parms.put(CACHING_KEY, caching); } @Override public void init() { - /* empty */ + /* empty */ } - + @Override public void dispose() { diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/context/AuthContext.java b/dev/cosbench-api/src/com/intel/cosbench/api/context/AuthContext.java index 40c8ef76..89eec8e3 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/context/AuthContext.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/context/AuthContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.context; @@ -23,22 +23,22 @@ */ public abstract class AuthContext extends Context { - /** - * Generate unique ID for the authentication context with attached metadata, normally, it will - * extract some key-value pairs from the context to construct the ID. - * - * @param meta - * @return an unique ID - */ - public abstract String getID(String meta) ; - - - /** - * Generate unique ID for authentication context, normally, it will extract some key-value pairs - * from the context to construct the ID. - * - * @return an unique ID - */ - public abstract String getID(); - + /** + * Generate unique ID for the authentication context with attached metadata, normally, it will + * extract some key-value pairs from the context to construct the ID. + * + * @param meta + * @return an unique ID + */ + public abstract String getID(String meta) ; + + + /** + * Generate unique ID for authentication context, normally, it will extract some key-value pairs + * from the context to construct the ID. + * + * @return an unique ID + */ + public abstract String getID(); + } diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/context/Context.java b/dev/cosbench-api/src/com/intel/cosbench/api/context/Context.java index 92980905..eb1986b0 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/context/Context.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/context/Context.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.context; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/context/DefaultAuthContext.java b/dev/cosbench-api/src/com/intel/cosbench/api/context/DefaultAuthContext.java index 5455b013..598d4b8d 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/context/DefaultAuthContext.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/context/DefaultAuthContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,25 +12,25 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.context; /** * This class represents a default implementation of AuthContext. - * + * * @author ywang19 - * + * */ public class DefaultAuthContext extends AuthContext { - - public String getID(String meta) { - return this.toString(); + + public String getID(String meta) { + return this.toString(); + } + + public String getID() { + return this.toString(); } - - public String getID() { - return this.toString(); - } } diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/storage/NoneStorage.java b/dev/cosbench-api/src/com/intel/cosbench/api/storage/NoneStorage.java index 392e48e7..a7da2d88 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/storage/NoneStorage.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/storage/NoneStorage.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.storage; @@ -30,9 +30,9 @@ /** * This class encapsulates one none storage system which is used if no any other * storage system is assigned. - * + * * @author ywang19, qzheng7 - * + * */ public class NoneStorage implements StorageAPI { @@ -52,7 +52,7 @@ public NoneStorage() { public void init(Config config, Logger logger) { this.logger = logger; this.parms = new Context(); - + logging = config.getBoolean(LOGGING_KEY, LOGGING_DEFAULT); /* register all parameters */ parms.put(LOGGING_KEY, logging); @@ -61,15 +61,15 @@ public void init(Config config, Logger logger) { @Override public void setAuthContext(AuthContext info) { - setAuthFlag(true); + setAuthFlag(true); /* empty */ } - @Override - public AuthContext getAuthContext() { - // TODO Auto-generated method stub - return null; - } + @Override + public AuthContext getAuthContext() { + // TODO Auto-generated method stub + return null; + } @Override public void dispose() { @@ -92,10 +92,10 @@ public InputStream getObject(String container, String object, Config config) { logger.info("performing GET at /{}/{}", container, object); return new ByteArrayInputStream(new byte[] {}); } - + @Override public InputStream getList(String container, String object, Config config) { - if (logging) + if (logging) logger.info("performing LIST at /{}/{}", container, object); return new ByteArrayInputStream(new byte[] {}); } @@ -145,9 +145,9 @@ protected Map getMetadata(String container, String object, return Collections.emptyMap(); } public void setAuthFlag(Boolean auth) { - this.authFlag = auth; + this.authFlag = auth; } public Boolean isAuthValid() { - return authFlag; + return authFlag; } } diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPI.java b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPI.java index 7341d5bb..f0539f65 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPI.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPI.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.storage; @@ -29,7 +29,7 @@ public interface StorageAPI { * Initializes a Storage-API with parameters contained in the * given config, whose content depends on the specific storage * type. Normally, it will also initialize one client for storage access. - * + * * @param config * - one instance from com.intel.cosbench.config.Config, which * includes parameters for authentication, and it will be passed @@ -48,7 +48,7 @@ public interface StorageAPI { /** * Retrieves parameters and current settings used by the Storage-API. - * + * * @return Context - one Context instance which contains all parameters * configured for the storage. */ @@ -56,8 +56,8 @@ public interface StorageAPI { /** * Aborts the execution of an on-going storage operation (HTTP request) if - * there is one. - * The method expects to provide one approach to abort outstanding operations gracefully + * there is one. + * The method expects to provide one approach to abort outstanding operations gracefully * when the worker hits some termination criteria. */ public void abort(); @@ -65,7 +65,7 @@ public interface StorageAPI { /** * Associates authentication context with this Storage-API for further * storage operations. - * + * * @param info * - one AuthContext instance, normally, it's the result returned * by the login() from the Auth-API. @@ -79,7 +79,7 @@ public interface StorageAPI { /** * Downloads an object from a container. - * + * * @param container * - the name of a container. * @param object @@ -91,7 +91,7 @@ public interface StorageAPI { /** * Gets a list of containers/objects - * + * * @param container * - the name of a container. * @param object @@ -100,10 +100,10 @@ public interface StorageAPI { * - the configuration used for this operation. */ public InputStream getList(String container, String object, Config config); - + /** * Creates a new container. - * + * * @param container * - the name of a container. * @param config @@ -113,7 +113,7 @@ public interface StorageAPI { /** * Uploads an object into a given container. - * + * * @param container * - the name of a container. * @param object @@ -130,7 +130,7 @@ public void createObject(String container, String object, InputStream data, /** * Removes a given container. - * + * * @param container * - the name of a container to be removed. * @param config @@ -140,7 +140,7 @@ public void createObject(String container, String object, InputStream data, /** * Deletes a given object. - * + * * @param container * - the name of a container. * @param object @@ -152,19 +152,19 @@ public void createObject(String container, String object, InputStream data, // public Map getMetadata(String container, String object, // Config config); -// +// // public void createMetadata(String container, String object, Map map, Config config); - + /** - * set the current authorization validity + * set the current authorization validity * @param auth * - if the current authorization valid or not - */ + */ public void setAuthFlag(Boolean auth); /** - * check if the current authorization valid - * + * check if the current authorization valid + * */ public Boolean isAuthValid(); diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPIFactory.java b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPIFactory.java index ef32f0e2..36d3cb08 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPIFactory.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPIFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.storage; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPIService.java b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPIService.java index 4cfa1bae..f02050e8 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPIService.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageAPIService.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.storage; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageConstants.java b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageConstants.java index b539ee1c..c54fab9b 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageConstants.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageConstants.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.storage; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageException.java b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageException.java index 137c2779..ed931e89 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageException.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.storage; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageInterruptedException.java b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageInterruptedException.java index 52513e2e..f5235114 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageInterruptedException.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageInterruptedException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.storage; diff --git a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageTimeoutException.java b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageTimeoutException.java index 25a6de94..30018d97 100644 --- a/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageTimeoutException.java +++ b/dev/cosbench-api/src/com/intel/cosbench/api/storage/StorageTimeoutException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.storage; diff --git a/dev/cosbench-castor/META-INF/MANIFEST.MF b/dev/cosbench-castor/META-INF/MANIFEST.MF index d83dd54c..ec94c97f 100644 --- a/dev/cosbench-castor/META-INF/MANIFEST.MF +++ b/dev/cosbench-castor/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Castor Configuration Bundle Bundle-SymbolicName: cosbench-castor -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Fragment-Host: com.springsource.org.castor;bundle-version="[1.2.0,2.0.0)" Bundle-RequiredExecutionEnvironment: JavaSE-1.6 diff --git a/dev/cosbench-cdmi-base/META-INF/MANIFEST.MF b/dev/cosbench-cdmi-base/META-INF/MANIFEST.MF index 51462c07..af71b9c9 100644 --- a/dev/cosbench-cdmi-base/META-INF/MANIFEST.MF +++ b/dev/cosbench-cdmi-base/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: CDMI Base Client Bundle Bundle-SymbolicName: cosbench-cdmi-base -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/api/cdmi/base/CDMIStorage.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/api/cdmi/base/CDMIStorage.java index f1d47ddf..32bafd3a 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/api/cdmi/base/CDMIStorage.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/api/cdmi/base/CDMIStorage.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.cdmi.base; import static com.intel.cosbench.client.cdmi.base.CdmiConstants.*; @@ -16,21 +34,21 @@ public class CDMIStorage extends NoneStorage { - // below parameters expect to get from auth module. + // below parameters expect to get from auth module. private HttpClient httpClient; private String url; // below parameters expect to get from configuration file. private int timeout; private String rootPath; - private String type; + private String type; private String headers; private boolean flag; - + // local variables private BaseCdmiClient client; private String[] header_list; - + @Override public void init(Config config, Logger logger) { super.init(config, logger); @@ -40,38 +58,38 @@ public void init(Config config, Logger logger) { } private void initParms(Config config) { - rootPath = config.get(ROOT_PATH_KEY, ROOT_PATH_DEFAULT); + rootPath = config.get(ROOT_PATH_KEY, ROOT_PATH_DEFAULT); timeout = config.getInt(TIMEOUT_KEY, TIMEOUT_DEFAULT); headers = config.get(CUSTOM_HEADERS_KEY, CUSTOM_HEADERS_DEFAULT); flag = config.getBoolean(RAISE_DELETE_ERRORS_KEY, RAISE_DELETE_ERRORS_DEFAULT); type = config.get(CDMI_CONTENT_TYPE_KEY, CDMI_CONTENT_TYPE_DEFAULT); header_list = headers.split(","); - + parms.put(ROOT_PATH_KEY, rootPath); - parms.put(TIMEOUT_KEY, timeout); - parms.put(RAISE_DELETE_ERRORS_KEY, flag); - parms.put(CDMI_CONTENT_TYPE_KEY, type); + parms.put(TIMEOUT_KEY, timeout); + parms.put(RAISE_DELETE_ERRORS_KEY, flag); + parms.put(CDMI_CONTENT_TYPE_KEY, type); } @Override public void setAuthContext(AuthContext info) { super.setAuthContext(info); try { - httpClient = (HttpClient) info.get(AUTH_CLIENT_KEY); - if(httpClient == null) // client is set - httpClient = HttpClientUtil.createHttpClient(timeout); - url = info.getStr(STORAGE_URL_KEY) + rootPath; - + httpClient = (HttpClient) info.get(AUTH_CLIENT_KEY); + if(httpClient == null) // client is set + httpClient = HttpClientUtil.createHttpClient(timeout); + url = info.getStr(STORAGE_URL_KEY) + rootPath; + // subtitute headers // "headers=X-AUTH-TOKEN:;" Map headerKV = new HashMap(); for(String header : header_list) { - String[] kv = header.split(":"); - if (kv.length >= 2) { - headerKV.put(kv[0], info.getStr(kv[1])); - } + String[] kv = header.split(":"); + if (kv.length >= 2) { + headerKV.put(kv[0], info.getStr(kv[1])); + } } - + logger.debug("httpclient =" + httpClient + ", url = " + url); client.init(httpClient, url, headerKV, false); } catch (Exception e) { diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/api/cdmi/base/CDMIStorageFactory.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/api/cdmi/base/CDMIStorageFactory.java index f2d6635b..07dc3948 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/api/cdmi/base/CDMIStorageFactory.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/api/cdmi/base/CDMIStorageFactory.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.cdmi.base; import com.intel.cosbench.api.storage.*; diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/BaseCdmiClient.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/BaseCdmiClient.java index 8b4930de..c2faf71f 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/BaseCdmiClient.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/BaseCdmiClient.java @@ -1,51 +1,69 @@ -package com.intel.cosbench.client.cdmi.base; - -import java.io.*; -import java.util.*; - -import org.apache.http.Header; -import org.apache.http.auth.params.AuthPNames; -import org.apache.http.client.*; -import org.apache.http.client.params.AuthPolicy; -import org.apache.http.client.protocol.ClientContext; -import org.apache.http.impl.client.BasicAuthCache; -import org.apache.http.message.BasicHeader; -import org.apache.http.protocol.*; - - -public abstract class BaseCdmiClient { - protected boolean raise_delete_errors = false; - protected HttpClient client; - protected HttpContext httpContext; - protected String uri; - protected ArrayList
custom_headers = new ArrayList
(); -// -// public BaseCdmiClient() { -// this.raise_delete_errors = flag; -// } - - public void init(HttpClient httpClient, String uri, Map headerKV, boolean flag) { - this.client = httpClient; - this.httpContext = new BasicHttpContext(); - httpContext.setAttribute(AuthPNames.TARGET_AUTH_PREF, Arrays.asList(new String[] {AuthPolicy.BASIC, AuthPolicy.DIGEST})); - - final AuthCache authCache = new BasicAuthCache(); - httpContext.setAttribute(ClientContext.AUTH_CACHE, authCache); - this.uri = uri; - this.raise_delete_errors = flag; - - for(String key: headerKV.keySet()) - this.custom_headers.add(new BasicHeader(key, headerKV.get(key))); - } - - public abstract void dispose(); - - public abstract void createContainer(String container) throws IOException, CdmiException; - public abstract void deleteContainer(String container) throws IOException, CdmiException; - public abstract InputStream getObjectAsStream(String container, String object) throws IOException, CdmiException; - - public abstract void storeStreamedObject(String container, String object, - InputStream data, long length) throws IOException, CdmiClientException; - public abstract void deleteObject(String container, String object) - throws IOException, CdmiException; -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.client.cdmi.base; + +import java.io.*; +import java.util.*; + +import org.apache.http.Header; +import org.apache.http.auth.params.AuthPNames; +import org.apache.http.client.*; +import org.apache.http.client.params.AuthPolicy; +import org.apache.http.client.protocol.ClientContext; +import org.apache.http.impl.client.BasicAuthCache; +import org.apache.http.message.BasicHeader; +import org.apache.http.protocol.*; + + +public abstract class BaseCdmiClient { + protected boolean raise_delete_errors = false; + protected HttpClient client; + protected HttpContext httpContext; + protected String uri; + protected ArrayList
custom_headers = new ArrayList
(); +// +// public BaseCdmiClient() { +// this.raise_delete_errors = flag; +// } + + public void init(HttpClient httpClient, String uri, Map headerKV, boolean flag) { + this.client = httpClient; + this.httpContext = new BasicHttpContext(); + httpContext.setAttribute(AuthPNames.TARGET_AUTH_PREF, Arrays.asList(new String[] {AuthPolicy.BASIC, AuthPolicy.DIGEST})); + + final AuthCache authCache = new BasicAuthCache(); + httpContext.setAttribute(ClientContext.AUTH_CACHE, authCache); + this.uri = uri; + this.raise_delete_errors = flag; + + for(String key: headerKV.keySet()) + this.custom_headers.add(new BasicHeader(key, headerKV.get(key))); + } + + public abstract void dispose(); + + public abstract void createContainer(String container) throws IOException, CdmiException; + public abstract void deleteContainer(String container) throws IOException, CdmiException; + public abstract InputStream getObjectAsStream(String container, String object) throws IOException, CdmiException; + + public abstract void storeStreamedObject(String container, String object, + InputStream data, long length) throws IOException, CdmiClientException; + public abstract void deleteObject(String container, String object) + throws IOException, CdmiException; +} diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClient.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClient.java index ea82bd13..54f83b04 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClient.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClient.java @@ -1,252 +1,270 @@ -package com.intel.cosbench.client.cdmi.base; - -import static org.apache.http.HttpStatus.*; - -import java.io.*; - -import org.apache.commons.codec.EncoderException; -import org.apache.commons.codec.net.URLCodec; -import org.apache.http.*; -import org.apache.http.client.methods.*; -import org.apache.http.entity.StringEntity; -import org.apache.http.util.*; - -import com.intel.cosbench.client.cdmi.util.CdmiJsonInputStreamEntity; - -/** - * This class encapsulates operations to access cdmi compatible server with cdmi content type. - * - * @author ywang19 - * - */ -public class CdmiClient extends BaseCdmiClient { -// private boolean raise_delete_errors = false; -// private HttpClient client; -// private String uri; -// private ArrayList
custom_headers = new ArrayList
(); - - private final static String cdmi_ver = "1.0.1"; - -// public CdmiClient(boolean flag) { -// this.raise_delete_errors = flag; -// } - - public CdmiClient() { - super(); - } - -// public void init(HttpClient httpClient, String uri, Map headerKV, boolean flag) { -// this.client = httpClient; -// this.uri = uri; -// this.raise_delete_errors = flag; -// -// for(String key: headerKV.keySet()) -// this.custom_headers.add(new BasicHeader(key, headerKV.get(key))); -// } - - public void dispose() { - client.getConnectionManager().shutdown(); - } - - private void setCustomHeaders(HttpRequest method) { - for(Header header : custom_headers) - method.setHeader(header); - } - - public void createContainer(String container) throws IOException, - CdmiException { - HttpResponse response = null; - try { - // Create the request - HttpPut method = new HttpPut(uri + "/" + encodeURL(container) + "/"); - - method.setHeader("Accept", "application/cdmi-container"); - method.setHeader("Content-Type", "application/cdmi-container"); - method.setHeader("X-CDMI-Specification-Version", cdmi_ver); - setCustomHeaders(method); - - method.setEntity(new StringEntity("{}")); - - response = client.execute(method, httpContext); - int statusCode = response.getStatusLine().getStatusCode(); - - if (statusCode == SC_CREATED || statusCode == SC_ACCEPTED) { - return; - } - throw new CdmiException("unexpected return from server", - response.getAllHeaders(), response.getStatusLine()); - }finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - } - - public void deleteContainer(String container) throws IOException, - CdmiException { - // add storage access logic here. - HttpResponse response = null; - try { - // Create the request - HttpDelete method = new HttpDelete(uri + "/" + encodeURL(container) + "/"); // "http://localhost:8080/cdmi-server/TestContainer/"); - - method.setHeader("X-CDMI-Specification-Version", cdmi_ver); - setCustomHeaders(method); - - response = client.execute(method, httpContext); - int statusCode = response.getStatusLine().getStatusCode(); - if (statusCode == SC_NO_CONTENT) - return; - if (statusCode == SC_NOT_FOUND) - throw new CdmiFileNotFoundException("container not found: " - + container, response.getAllHeaders(), - response.getStatusLine()); - if (statusCode == SC_CONFLICT) - throw new CdmiConflictException( - "cannot delete an non-empty container", - response.getAllHeaders(), response.getStatusLine()); - throw new CdmiException("unexpected return from server", - response.getAllHeaders(), response.getStatusLine()); - } finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - } - - public InputStream getObjectAsStream(String container, String object) - throws IOException, CdmiException { - - HttpResponse response = null; - // Create the request - HttpGet method = new HttpGet(uri + "/" + encodeURL(container) - + "/" + encodeURL(object)); - - method.setHeader("Accept", "application/cdmi-object"); - method.setHeader("X-CDMI-Specification-Version", cdmi_ver); - setCustomHeaders(method); - - response = client.execute(method, httpContext); - int statusCode = response.getStatusLine().getStatusCode(); - if (statusCode == SC_OK) - return response.getEntity().getContent(); - - if (statusCode == SC_NOT_FOUND) - throw new CdmiFileNotFoundException("object not found: " - + container + "/" + object, response.getAllHeaders(), - response.getStatusLine()); - throw new CdmiException("unexpected result from server", - response.getAllHeaders(), response.getStatusLine()); - } - - @SuppressWarnings("unused") - private void dumpMethod(HttpRequestBase method) { - System.out.println("==== METHOD BEGIN ===="); - System.out.println(method.getMethod()); - System.out.println(method.getURI()); - for(Header header: method.getAllHeaders()) { - System.out.println(header.getName() + ": " + header.getValue()); - } - System.out.println("==== METHOD END ===="); - } - - @SuppressWarnings("unused") - private void dumpResponse(HttpResponse response) { - System.out.println("==== RESPONSE BEGIN ===="); - Header[] hdr = response.getAllHeaders(); - System.out.println("Headers : " + hdr.length); - for (int i = 0; i < hdr.length; i++) { - System.out.println(hdr[i]); - } - System.out.println("---------"); - System.out.println(response.getProtocolVersion()); - System.out.println(response.getStatusLine().getStatusCode()); - - System.out.println(response.getStatusLine().getReasonPhrase()); - System.out.println(response.getStatusLine().toString()); - System.out.println("---------"); - System.out.println("==== RESPONSE END ===="); - } - - public void storeStreamedObject(String container, String object, - InputStream data, long length) throws IOException, CdmiClientException { - // add storage access logic here. - HttpPut method = null; - // Create the request - HttpResponse response = null; - try { - method = new HttpPut(uri + "/" + encodeURL(container) - + "/" + encodeURL(object)); - - method.setHeader("Accept", "application/cdmi-object"); - method.setHeader("Content-Type", "application/cdmi-object"); - method.setHeader("X-CDMI-Specification-Version", cdmi_ver); - setCustomHeaders(method); - - CdmiJsonInputStreamEntity entity = new CdmiJsonInputStreamEntity(data, length); - - if (length < 0) - entity.setChunked(true); - else { - entity.setChunked(false); - } - - method.setEntity(entity); - - response = client.execute(method, httpContext); - int statusCode = response.getStatusLine().getStatusCode(); - if (statusCode == HttpStatus.SC_CREATED) { - return; - } if (statusCode == SC_ACCEPTED) - return; - if (statusCode == SC_NOT_FOUND) - throw new FileNotFoundException("container not found: " - + container); - else { - throw new CdmiClientException(statusCode, "Unexpected Server Response: " - + response.getStatusLine(), response.getAllHeaders(), - response.getStatusLine()); - } - }finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - } - - public void deleteObject(String container, String object) - throws IOException, CdmiException { - HttpResponse response = null; - try { - // Create the request - HttpDelete method = new HttpDelete(uri + "/" + encodeURL(container) - + "/" + encodeURL(object)); - - method.setHeader("X-CDMI-Specification-Version", cdmi_ver); - setCustomHeaders(method); - - response = client.execute(method, httpContext); - - int statusCode = response.getStatusLine().getStatusCode(); - if (statusCode == SC_NO_CONTENT) - return; - if (!raise_delete_errors) - return; - if (statusCode == SC_NOT_FOUND) - throw new CdmiFileNotFoundException("object not found: " - + container + "/" + object, - response.getAllHeaders(), response.getStatusLine()); - throw new CdmiException("unexpected return from server", - response.getAllHeaders(), response.getStatusLine()); - } finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - } - - public static String encodeURL(String str) { - URLCodec codec = new URLCodec(); - try { - return codec.encode(str).replaceAll("\\+", "%20"); - } catch (EncoderException ee) { - return str; - } - } -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.client.cdmi.base; + +import static org.apache.http.HttpStatus.*; + +import java.io.*; + +import org.apache.commons.codec.EncoderException; +import org.apache.commons.codec.net.URLCodec; +import org.apache.http.*; +import org.apache.http.client.methods.*; +import org.apache.http.entity.StringEntity; +import org.apache.http.util.*; + +import com.intel.cosbench.client.cdmi.util.CdmiJsonInputStreamEntity; + +/** + * This class encapsulates operations to access cdmi compatible server with cdmi content type. + * + * @author ywang19 + * + */ +public class CdmiClient extends BaseCdmiClient { +// private boolean raise_delete_errors = false; +// private HttpClient client; +// private String uri; +// private ArrayList
custom_headers = new ArrayList
(); + + private final static String cdmi_ver = "1.0.1"; + +// public CdmiClient(boolean flag) { +// this.raise_delete_errors = flag; +// } + + public CdmiClient() { + super(); + } + +// public void init(HttpClient httpClient, String uri, Map headerKV, boolean flag) { +// this.client = httpClient; +// this.uri = uri; +// this.raise_delete_errors = flag; +// +// for(String key: headerKV.keySet()) +// this.custom_headers.add(new BasicHeader(key, headerKV.get(key))); +// } + + public void dispose() { + client.getConnectionManager().shutdown(); + } + + private void setCustomHeaders(HttpRequest method) { + for(Header header : custom_headers) + method.setHeader(header); + } + + public void createContainer(String container) throws IOException, + CdmiException { + HttpResponse response = null; + try { + // Create the request + HttpPut method = new HttpPut(uri + "/" + encodeURL(container) + "/"); + + method.setHeader("Accept", "application/cdmi-container"); + method.setHeader("Content-Type", "application/cdmi-container"); + method.setHeader("X-CDMI-Specification-Version", cdmi_ver); + setCustomHeaders(method); + + method.setEntity(new StringEntity("{}")); + + response = client.execute(method, httpContext); + int statusCode = response.getStatusLine().getStatusCode(); + + if (statusCode == SC_CREATED || statusCode == SC_ACCEPTED) { + return; + } + throw new CdmiException("unexpected return from server", + response.getAllHeaders(), response.getStatusLine()); + }finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + } + + public void deleteContainer(String container) throws IOException, + CdmiException { + // add storage access logic here. + HttpResponse response = null; + try { + // Create the request + HttpDelete method = new HttpDelete(uri + "/" + encodeURL(container) + "/"); // "http://localhost:8080/cdmi-server/TestContainer/"); + + method.setHeader("X-CDMI-Specification-Version", cdmi_ver); + setCustomHeaders(method); + + response = client.execute(method, httpContext); + int statusCode = response.getStatusLine().getStatusCode(); + if (statusCode == SC_NO_CONTENT) + return; + if (statusCode == SC_NOT_FOUND) + throw new CdmiFileNotFoundException("container not found: " + + container, response.getAllHeaders(), + response.getStatusLine()); + if (statusCode == SC_CONFLICT) + throw new CdmiConflictException( + "cannot delete an non-empty container", + response.getAllHeaders(), response.getStatusLine()); + throw new CdmiException("unexpected return from server", + response.getAllHeaders(), response.getStatusLine()); + } finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + } + + public InputStream getObjectAsStream(String container, String object) + throws IOException, CdmiException { + + HttpResponse response = null; + // Create the request + HttpGet method = new HttpGet(uri + "/" + encodeURL(container) + + "/" + encodeURL(object)); + + method.setHeader("Accept", "application/cdmi-object"); + method.setHeader("X-CDMI-Specification-Version", cdmi_ver); + setCustomHeaders(method); + + response = client.execute(method, httpContext); + int statusCode = response.getStatusLine().getStatusCode(); + if (statusCode == SC_OK) + return response.getEntity().getContent(); + + if (statusCode == SC_NOT_FOUND) + throw new CdmiFileNotFoundException("object not found: " + + container + "/" + object, response.getAllHeaders(), + response.getStatusLine()); + throw new CdmiException("unexpected result from server", + response.getAllHeaders(), response.getStatusLine()); + } + + @SuppressWarnings("unused") + private void dumpMethod(HttpRequestBase method) { + System.out.println("==== METHOD BEGIN ===="); + System.out.println(method.getMethod()); + System.out.println(method.getURI()); + for(Header header: method.getAllHeaders()) { + System.out.println(header.getName() + ": " + header.getValue()); + } + System.out.println("==== METHOD END ===="); + } + + @SuppressWarnings("unused") + private void dumpResponse(HttpResponse response) { + System.out.println("==== RESPONSE BEGIN ===="); + Header[] hdr = response.getAllHeaders(); + System.out.println("Headers : " + hdr.length); + for (int i = 0; i < hdr.length; i++) { + System.out.println(hdr[i]); + } + System.out.println("---------"); + System.out.println(response.getProtocolVersion()); + System.out.println(response.getStatusLine().getStatusCode()); + + System.out.println(response.getStatusLine().getReasonPhrase()); + System.out.println(response.getStatusLine().toString()); + System.out.println("---------"); + System.out.println("==== RESPONSE END ===="); + } + + public void storeStreamedObject(String container, String object, + InputStream data, long length) throws IOException, CdmiClientException { + // add storage access logic here. + HttpPut method = null; + // Create the request + HttpResponse response = null; + try { + method = new HttpPut(uri + "/" + encodeURL(container) + + "/" + encodeURL(object)); + + method.setHeader("Accept", "application/cdmi-object"); + method.setHeader("Content-Type", "application/cdmi-object"); + method.setHeader("X-CDMI-Specification-Version", cdmi_ver); + setCustomHeaders(method); + + CdmiJsonInputStreamEntity entity = new CdmiJsonInputStreamEntity(data, length); + + if (length < 0) + entity.setChunked(true); + else { + entity.setChunked(false); + } + + method.setEntity(entity); + + response = client.execute(method, httpContext); + int statusCode = response.getStatusLine().getStatusCode(); + if (statusCode == HttpStatus.SC_CREATED) { + return; + } if (statusCode == SC_ACCEPTED) + return; + if (statusCode == SC_NOT_FOUND) + throw new FileNotFoundException("container not found: " + + container); + else { + throw new CdmiClientException(statusCode, "Unexpected Server Response: " + + response.getStatusLine(), response.getAllHeaders(), + response.getStatusLine()); + } + }finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + } + + public void deleteObject(String container, String object) + throws IOException, CdmiException { + HttpResponse response = null; + try { + // Create the request + HttpDelete method = new HttpDelete(uri + "/" + encodeURL(container) + + "/" + encodeURL(object)); + + method.setHeader("X-CDMI-Specification-Version", cdmi_ver); + setCustomHeaders(method); + + response = client.execute(method, httpContext); + + int statusCode = response.getStatusLine().getStatusCode(); + if (statusCode == SC_NO_CONTENT) + return; + if (!raise_delete_errors) + return; + if (statusCode == SC_NOT_FOUND) + throw new CdmiFileNotFoundException("object not found: " + + container + "/" + object, + response.getAllHeaders(), response.getStatusLine()); + throw new CdmiException("unexpected return from server", + response.getAllHeaders(), response.getStatusLine()); + } finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + } + + public static String encodeURL(String str) { + URLCodec codec = new URLCodec(); + try { + return codec.encode(str).replaceAll("\\+", "%20"); + } catch (EncoderException ee) { + return str; + } + } +} diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClientException.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClientException.java index 7fe9d6a0..08e53fb4 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClientException.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClientException.java @@ -1,33 +1,51 @@ -package com.intel.cosbench.client.cdmi.base; - -import org.apache.http.Header; -import org.apache.http.StatusLine; -import org.apache.http.client.HttpResponseException; - - -public class CdmiClientException extends HttpResponseException { - private Header[] httpHeaders; - private StatusLine httpStatusLine; - - public CdmiClientException(int code, String message) { - super(code, message); - } - - public CdmiClientException(int code, String message, Header[] httpHeaders, - StatusLine httpStatusLine) { - super(code, message); - this.httpHeaders = httpHeaders; - this.httpStatusLine = httpStatusLine; - } - - public Header[] getHttpHeaders() { - return httpHeaders; - } - - public StatusLine getHttpStatusLine() { - return httpStatusLine; - } - - private static final long serialVersionUID = 1L; - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.client.cdmi.base; + +import org.apache.http.Header; +import org.apache.http.StatusLine; +import org.apache.http.client.HttpResponseException; + + +public class CdmiClientException extends HttpResponseException { + private Header[] httpHeaders; + private StatusLine httpStatusLine; + + public CdmiClientException(int code, String message) { + super(code, message); + } + + public CdmiClientException(int code, String message, Header[] httpHeaders, + StatusLine httpStatusLine) { + super(code, message); + this.httpHeaders = httpHeaders; + this.httpStatusLine = httpStatusLine; + } + + public Header[] getHttpHeaders() { + return httpHeaders; + } + + public StatusLine getHttpStatusLine() { + return httpStatusLine; + } + + private static final long serialVersionUID = 1L; + +} diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClientFactory.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClientFactory.java index 5789973d..d2c85af4 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClientFactory.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiClientFactory.java @@ -1,21 +1,39 @@ -package com.intel.cosbench.client.cdmi.base; - -/** - * This class provides a method to create corresponding client to access cdmi compatible server in different form (like cdmi content type or non-cdmi content type). - * - * @author ywang19 - * - */ -public class CdmiClientFactory { - - public static BaseCdmiClient getClient(String type) { - if("cdmi".equalsIgnoreCase(type)) { // cdmi content type - return new CdmiClient(); - }else if("non-cdmi".equalsIgnoreCase(type)) { - return new NonCdmiClient(); - }else { - System.err.println("Type: " + type + " is not supported yet."); - return null; - } - } -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.client.cdmi.base; + +/** + * This class provides a method to create corresponding client to access cdmi compatible server in different form (like cdmi content type or non-cdmi content type). + * + * @author ywang19 + * + */ +public class CdmiClientFactory { + + public static BaseCdmiClient getClient(String type) { + if("cdmi".equalsIgnoreCase(type)) { // cdmi content type + return new CdmiClient(); + }else if("non-cdmi".equalsIgnoreCase(type)) { + return new NonCdmiClient(); + }else { + System.err.println("Type: " + type + " is not supported yet."); + return null; + } + } +} diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiConflictException.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiConflictException.java index e75bc973..2272f0a2 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiConflictException.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiConflictException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.cdmi.base; diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiConstants.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiConstants.java index 1f45a8ed..67abde6e 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiConstants.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiConstants.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.client.cdmi.base; public interface CdmiConstants { @@ -10,16 +28,16 @@ public interface CdmiConstants { String ROOT_PATH_KEY = "root_path"; String ROOT_PATH_DEFAULT = ""; - + String RAISE_DELETE_ERRORS_KEY = "raise_delete_errors"; Boolean RAISE_DELETE_ERRORS_DEFAULT = false; - + String CUSTOM_HEADERS_KEY = "custom_headers"; String CUSTOM_HEADERS_DEFAULT = ""; - + String CDMI_CONTENT_TYPE_KEY = "type"; String CDMI_CONTENT_TYPE_DEFAULT = "cdmi"; - + // -------------------------------------------------------------------------- // CONNECTION // -------------------------------------------------------------------------- diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiException.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiException.java index 711d807c..4fdfe7f4 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiException.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.cdmi.base; diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiFileNotFoundException.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiFileNotFoundException.java index c59f9595..e125e57d 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiFileNotFoundException.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/CdmiFileNotFoundException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.cdmi.base; diff --git a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/NonCdmiClient.java b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/NonCdmiClient.java index b1a249aa..26c695b2 100644 --- a/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/NonCdmiClient.java +++ b/dev/cosbench-cdmi-base/src/com/intel/cosbench/client/cdmi/base/NonCdmiClient.java @@ -1,230 +1,248 @@ -package com.intel.cosbench.client.cdmi.base; - -import static org.apache.http.HttpStatus.*; - -import java.io.*; - -import org.apache.commons.codec.EncoderException; -import org.apache.commons.codec.net.URLCodec; -import org.apache.http.*; -import org.apache.http.client.methods.*; -import org.apache.http.entity.InputStreamEntity; -import org.apache.http.util.*; - - -/** - * This class encapsulates operations to access cdmi compatible server with non-cdmi content type. - * - * @author ywang19 - * - */ -public class NonCdmiClient extends BaseCdmiClient{ -// private HttpClient client; -// private String uri; -// private ArrayList
custom_headers = new ArrayList
(); - - public NonCdmiClient() { - super(); - } - -// public void init(HttpClient httpClient, String uri, Map headerKV) { -// this.client = httpClient; -// this.uri = uri; -// -// for(String key: headerKV.keySet()) -// this.custom_headers.add(new BasicHeader(key, headerKV.get(key))); -// } - - public void dispose() { - client.getConnectionManager().shutdown(); - } - - private void setCustomHeaders(HttpRequest method) { - for(Header header : custom_headers) - method.setHeader(header); - } - - public void createContainer(String container) throws IOException, - CdmiException { - HttpResponse response = null; - try { - // Create the request - HttpPut method = new HttpPut(uri + "/" + encodeURL(container) + "/"); - - setCustomHeaders(method); - - response = client.execute(method, httpContext); - int statusCode = response.getStatusLine().getStatusCode(); - - if (statusCode == SC_CREATED || statusCode == SC_ACCEPTED) { - return; - } - throw new CdmiException("unexpected return from server", - response.getAllHeaders(), response.getStatusLine()); - }finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - } - - public void deleteContainer(String container) throws IOException, - CdmiException { - // add storage access logic here. - HttpResponse response = null; - try { - // Create the request - HttpDelete method = new HttpDelete(uri + "/" + encodeURL(container) + "/"); // "http://localhost:8080/cdmi-server/TestContainer/"); - - setCustomHeaders(method); - - response = client.execute(method, httpContext); - int statusCode = response.getStatusLine().getStatusCode(); - if (statusCode == SC_NO_CONTENT) - return; - if (statusCode == SC_NOT_FOUND) - throw new CdmiFileNotFoundException("container not found: " - + container, response.getAllHeaders(), - response.getStatusLine()); - if (statusCode == SC_CONFLICT) - throw new CdmiConflictException( - "cannot delete an non-empty container", - response.getAllHeaders(), response.getStatusLine()); - throw new CdmiException("unexpected return from server", - response.getAllHeaders(), response.getStatusLine()); - } finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - } - - public InputStream getObjectAsStream(String container, String object) - throws IOException, CdmiException { - - HttpResponse response = null; - // Create the request - HttpGet method = new HttpGet(uri + "/" + encodeURL(container) - + "/" + encodeURL(object)); - - setCustomHeaders(method); - - response = client.execute(method, httpContext); - int statusCode = response.getStatusLine().getStatusCode(); - if (statusCode == SC_OK) - return response.getEntity().getContent(); - - if (statusCode == SC_NOT_FOUND) - throw new CdmiFileNotFoundException("object not found: " - + container + "/" + object, response.getAllHeaders(), - response.getStatusLine()); - throw new CdmiException("unexpected result from server", - response.getAllHeaders(), response.getStatusLine()); - } - - @SuppressWarnings("unused") - private void dumpMethod(HttpRequestBase method) { - System.out.println("==== METHOD BEGIN ===="); - System.out.println(method.getMethod()); - System.out.println(method.getURI()); - for(Header header: method.getAllHeaders()) { - System.out.println(header.getName() + ": " + header.getValue()); - } - System.out.println("==== METHOD END ===="); - } - - @SuppressWarnings("unused") - private void dumpResponse(HttpResponse response) { - System.out.println("==== RESPONSE BEGIN ===="); - Header[] hdr = response.getAllHeaders(); - System.out.println("Headers : " + hdr.length); - for (int i = 0; i < hdr.length; i++) { - System.out.println(hdr[i]); - } - System.out.println("---------"); - System.out.println(response.getProtocolVersion()); - System.out.println(response.getStatusLine().getStatusCode()); - - System.out.println(response.getStatusLine().getReasonPhrase()); - System.out.println(response.getStatusLine().toString()); - System.out.println("---------"); - System.out.println("==== RESPONSE END ===="); - } - - public void storeStreamedObject(String container, String object, - InputStream data, long length) throws IOException, CdmiClientException { - // add storage access logic here. - HttpPut method = null; - // Create the request - HttpResponse response = null; - try { - method = new HttpPut(uri + "/" + encodeURL(container) - + "/" + encodeURL(object)); - - method.setHeader("Content-Type", "application/octet-stream"); - setCustomHeaders(method); - InputStreamEntity entity = new InputStreamEntity(data, length); - if (length < 0) - entity.setChunked(true); - else { - entity.setChunked(false); - } - - method.setEntity(entity); - - response = client.execute(method, httpContext); - int statusCode = response.getStatusLine().getStatusCode(); - if (statusCode == HttpStatus.SC_CREATED) { - return; - } if (statusCode == SC_ACCEPTED) - return; - if (statusCode == SC_NOT_FOUND) - throw new FileNotFoundException("container not found: " - + container); - else { - throw new CdmiClientException(statusCode, "Unexpected Server Response: " - + response.getStatusLine(), response.getAllHeaders(), - response.getStatusLine()); - } - }finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - } - - public void deleteObject(String container, String object) - throws IOException, CdmiException { - HttpResponse response = null; - try { - // Create the request - HttpDelete method = new HttpDelete(uri + "/" + encodeURL(container) - + "/" + encodeURL(object)); - - setCustomHeaders(method); - - response = client.execute(method, httpContext); - - int statusCode = response.getStatusLine().getStatusCode(); - if (statusCode == SC_NO_CONTENT) - return; - if (!raise_delete_errors) - return; - if (statusCode == SC_NOT_FOUND) - throw new CdmiFileNotFoundException("object not found: " - + container + "/" + object, - response.getAllHeaders(), response.getStatusLine()); - throw new CdmiException("unexpected return from server", - response.getAllHeaders(), response.getStatusLine()); - } finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - } - - public static String encodeURL(String str) { - URLCodec codec = new URLCodec(); - try { - return codec.encode(str).replaceAll("\\+", "%20"); - } catch (EncoderException ee) { - return str; - } - } -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.client.cdmi.base; + +import static org.apache.http.HttpStatus.*; + +import java.io.*; + +import org.apache.commons.codec.EncoderException; +import org.apache.commons.codec.net.URLCodec; +import org.apache.http.*; +import org.apache.http.client.methods.*; +import org.apache.http.entity.InputStreamEntity; +import org.apache.http.util.*; + + +/** + * This class encapsulates operations to access cdmi compatible server with non-cdmi content type. + * + * @author ywang19 + * + */ +public class NonCdmiClient extends BaseCdmiClient{ +// private HttpClient client; +// private String uri; +// private ArrayList
custom_headers = new ArrayList
(); + + public NonCdmiClient() { + super(); + } + +// public void init(HttpClient httpClient, String uri, Map headerKV) { +// this.client = httpClient; +// this.uri = uri; +// +// for(String key: headerKV.keySet()) +// this.custom_headers.add(new BasicHeader(key, headerKV.get(key))); +// } + + public void dispose() { + client.getConnectionManager().shutdown(); + } + + private void setCustomHeaders(HttpRequest method) { + for(Header header : custom_headers) + method.setHeader(header); + } + + public void createContainer(String container) throws IOException, + CdmiException { + HttpResponse response = null; + try { + // Create the request + HttpPut method = new HttpPut(uri + "/" + encodeURL(container) + "/"); + + setCustomHeaders(method); + + response = client.execute(method, httpContext); + int statusCode = response.getStatusLine().getStatusCode(); + + if (statusCode == SC_CREATED || statusCode == SC_ACCEPTED) { + return; + } + throw new CdmiException("unexpected return from server", + response.getAllHeaders(), response.getStatusLine()); + }finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + } + + public void deleteContainer(String container) throws IOException, + CdmiException { + // add storage access logic here. + HttpResponse response = null; + try { + // Create the request + HttpDelete method = new HttpDelete(uri + "/" + encodeURL(container) + "/"); // "http://localhost:8080/cdmi-server/TestContainer/"); + + setCustomHeaders(method); + + response = client.execute(method, httpContext); + int statusCode = response.getStatusLine().getStatusCode(); + if (statusCode == SC_NO_CONTENT) + return; + if (statusCode == SC_NOT_FOUND) + throw new CdmiFileNotFoundException("container not found: " + + container, response.getAllHeaders(), + response.getStatusLine()); + if (statusCode == SC_CONFLICT) + throw new CdmiConflictException( + "cannot delete an non-empty container", + response.getAllHeaders(), response.getStatusLine()); + throw new CdmiException("unexpected return from server", + response.getAllHeaders(), response.getStatusLine()); + } finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + } + + public InputStream getObjectAsStream(String container, String object) + throws IOException, CdmiException { + + HttpResponse response = null; + // Create the request + HttpGet method = new HttpGet(uri + "/" + encodeURL(container) + + "/" + encodeURL(object)); + + setCustomHeaders(method); + + response = client.execute(method, httpContext); + int statusCode = response.getStatusLine().getStatusCode(); + if (statusCode == SC_OK) + return response.getEntity().getContent(); + + if (statusCode == SC_NOT_FOUND) + throw new CdmiFileNotFoundException("object not found: " + + container + "/" + object, response.getAllHeaders(), + response.getStatusLine()); + throw new CdmiException("unexpected result from server", + response.getAllHeaders(), response.getStatusLine()); + } + + @SuppressWarnings("unused") + private void dumpMethod(HttpRequestBase method) { + System.out.println("==== METHOD BEGIN ===="); + System.out.println(method.getMethod()); + System.out.println(method.getURI()); + for(Header header: method.getAllHeaders()) { + System.out.println(header.getName() + ": " + header.getValue()); + } + System.out.println("==== METHOD END ===="); + } + + @SuppressWarnings("unused") + private void dumpResponse(HttpResponse response) { + System.out.println("==== RESPONSE BEGIN ===="); + Header[] hdr = response.getAllHeaders(); + System.out.println("Headers : " + hdr.length); + for (int i = 0; i < hdr.length; i++) { + System.out.println(hdr[i]); + } + System.out.println("---------"); + System.out.println(response.getProtocolVersion()); + System.out.println(response.getStatusLine().getStatusCode()); + + System.out.println(response.getStatusLine().getReasonPhrase()); + System.out.println(response.getStatusLine().toString()); + System.out.println("---------"); + System.out.println("==== RESPONSE END ===="); + } + + public void storeStreamedObject(String container, String object, + InputStream data, long length) throws IOException, CdmiClientException { + // add storage access logic here. + HttpPut method = null; + // Create the request + HttpResponse response = null; + try { + method = new HttpPut(uri + "/" + encodeURL(container) + + "/" + encodeURL(object)); + + method.setHeader("Content-Type", "application/octet-stream"); + setCustomHeaders(method); + InputStreamEntity entity = new InputStreamEntity(data, length); + if (length < 0) + entity.setChunked(true); + else { + entity.setChunked(false); + } + + method.setEntity(entity); + + response = client.execute(method, httpContext); + int statusCode = response.getStatusLine().getStatusCode(); + if (statusCode == HttpStatus.SC_CREATED) { + return; + } if (statusCode == SC_ACCEPTED) + return; + if (statusCode == SC_NOT_FOUND) + throw new FileNotFoundException("container not found: " + + container); + else { + throw new CdmiClientException(statusCode, "Unexpected Server Response: " + + response.getStatusLine(), response.getAllHeaders(), + response.getStatusLine()); + } + }finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + } + + public void deleteObject(String container, String object) + throws IOException, CdmiException { + HttpResponse response = null; + try { + // Create the request + HttpDelete method = new HttpDelete(uri + "/" + encodeURL(container) + + "/" + encodeURL(object)); + + setCustomHeaders(method); + + response = client.execute(method, httpContext); + + int statusCode = response.getStatusLine().getStatusCode(); + if (statusCode == SC_NO_CONTENT) + return; + if (!raise_delete_errors) + return; + if (statusCode == SC_NOT_FOUND) + throw new CdmiFileNotFoundException("object not found: " + + container + "/" + object, + response.getAllHeaders(), response.getStatusLine()); + throw new CdmiException("unexpected return from server", + response.getAllHeaders(), response.getStatusLine()); + } finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + } + + public static String encodeURL(String str) { + URLCodec codec = new URLCodec(); + try { + return codec.encode(str).replaceAll("\\+", "%20"); + } catch (EncoderException ee) { + return str; + } + } +} diff --git a/dev/cosbench-cdmi-swift/META-INF/MANIFEST.MF b/dev/cosbench-cdmi-swift/META-INF/MANIFEST.MF index 209967d1..39f0feb1 100644 --- a/dev/cosbench-cdmi-swift/META-INF/MANIFEST.MF +++ b/dev/cosbench-cdmi-swift/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: CDMI Client for Swift Bundle Bundle-SymbolicName: cosbench-cdmi-swift -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-cdmi-swift/src/com/intel/cosbench/api/cdmiswift/CDMIStorage.java b/dev/cosbench-cdmi-swift/src/com/intel/cosbench/api/cdmiswift/CDMIStorage.java index edcf9f25..bc634934 100644 --- a/dev/cosbench-cdmi-swift/src/com/intel/cosbench/api/cdmiswift/CDMIStorage.java +++ b/dev/cosbench-cdmi-swift/src/com/intel/cosbench/api/cdmiswift/CDMIStorage.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.cdmiswift; import static com.intel.cosbench.client.cdmiswift.CdmiSwiftConstants.*; @@ -18,7 +36,7 @@ public class CDMIStorage extends NoneStorage { private CdmiSwiftClient client; private int timeout; - + private String rootPath; private String storageUrl; @@ -26,36 +44,36 @@ public class CDMIStorage extends NoneStorage { public void init(Config config, Logger logger) { super.init(config, logger); initParms(config); - + HttpClient httpClient = HttpClientUtil.createHttpClient(timeout); client = new CdmiSwiftClient(httpClient); } private void initParms(Config config) { - rootPath = config.get(ROOT_PATH_KEY, ROOT_PATH_DEFAULT); - storageUrl = config.get(STORAGE_URL_KEY, STORAGE_URL_DEFAULT); + rootPath = config.get(ROOT_PATH_KEY, ROOT_PATH_DEFAULT); + storageUrl = config.get(STORAGE_URL_KEY, STORAGE_URL_DEFAULT); timeout = config.getInt(TIMEOUT_KEY, TIMEOUT_DEFAULT); - + parms.put(ROOT_PATH_KEY, rootPath); parms.put(STORAGE_URL_KEY, storageUrl); - parms.put(TIMEOUT_KEY, timeout); + parms.put(TIMEOUT_KEY, timeout); } - + // @Override // public void setAuthContext(AuthContext info) { // super.setAuthContext(info); // try { -// String authToken = info.getStr(AUTH_TOKEN_KEY); +// String authToken = info.getStr(AUTH_TOKEN_KEY); // String tempUrl = info.getStr(STORAGE_URL_KEY); // String parts[] = tempUrl.split("/"); // if(parts.length > 2) // { -// parts[parts.length-2] = nsroot; -// tempUrl = ""; -// for(int i=0; i data = new TreeMap() ; - private TreeMap metadata = new TreeMap(); - - public CdmiJsonInputStreamEntity() - { - this.content = null; - this.length = -1; - this.transferLen = -1; - } - - public CdmiJsonInputStreamEntity(InputStream instream, long length) - { - content = instream; - this.length = length; - this.transferLen = -1; - } - - public boolean isRepeatable() - { - if(content == null) - return true; - return content.markSupported(); - } - - public long getContentLength() - { - return transferLen; - } - - public long getTransferLength() - { - return transferLen; - } - - public InputStream getContent() - throws IOException - { - return content; - } - - public void addMetadata(String key, String value) { - metadata.put(key, value); - } - - public void addField(String key, String value) { - data.put(key, value); - } - - public void addContent(InputStream content, long length) { - this.content = content; - this.length = length; - } - - private String encodeEntity() { - data.put("metadata", encodeJson(metadata)); - - if(content != null) { - addField("mimetype", "text/plain"); - addField("valuetransferencoding", "utf-8"); - } - - return encodeJson(data); - } - - private long writePre(OutputStream outstream) throws UnsupportedEncodingException, IOException { - StringBuffer buffer = new StringBuffer(encodeEntity()); - if(length > 0) { - buffer.replace(buffer.length() - 1, buffer.length(), ","); - buffer.append(" \"value\": \""); - } - byte[] bytes = buffer.toString().getBytes("utf-8"); - - outstream.write(bytes); - - return bytes.length; - } - - private long writePost(OutputStream outstream) throws UnsupportedEncodingException, IOException { - if(length > 0) { - StringBuffer buffer = new StringBuffer("\" }"); - byte[] bytes = buffer.toString().getBytes("UTF-8"); - - outstream.write(bytes); - outstream.flush(); - - return bytes.length; - }else { - outstream.flush(); - - return length; - } - } - - public boolean isChunked() - { - return false; - } - - public Header getContentType() - { - return new BasicHeader("Content-Type", "application/cdmi-object"); - } - - /** - * Write value stream to output stream - * - * @param instream - * @param outstream - * @return - * @throws IOException - */ - private long writeContent(InputStream instream, OutputStream outstream) - { - if(instream == null) - return 0L; - - byte buffer[] = new byte[BUFFER_SIZE]; - int l; - long valueLen = 0; - - try{ - if(length < 0L) { - while((l = instream.read(buffer)) != -1) { - outstream.write(buffer, 0, l); - valueLen += l; - } - } else { - long remaining = length; - do - { - if(remaining <= 0L) - break; - l = instream.read(buffer, 0, (int)Math.min(BUFFER_SIZE, remaining)); - if(l == -1) - break; - outstream.write(buffer, 0, l); - valueLen += l; - remaining -= l; - } while(true); - } - }catch(IOException e) { - e.printStackTrace(); - } - - return valueLen; - } - - public void writeTo(OutputStream outstream) throws IOException - { - transferLen = 0; - if(outstream == null) - throw new IllegalArgumentException("Output stream may not be null"); - InputStream instream = content; - - try { - transferLen += writePre(outstream); - transferLen += writeContent(instream, outstream); - transferLen += writePost(outstream); - }finally { - if(instream != null) - instream.close(); - } - } - - public boolean isStreaming() - { - return true; - } - - /** - * @deprecated Method consumeContent is deprecated - */ - - public void consumeContent() - throws IOException - { - content.close(); - } - - /** - * Encode all items except "value" into json format. - * - * @param entries - * @return - */ - private static String encodeJson(Map entries) { - boolean hasValue = false; - StringBuffer buffer = new StringBuffer(); - buffer.append('{'); - for (Map.Entry item : entries.entrySet()) { - // filter out "value" - if (item.getKey().equals("value")) { - continue; - } - - if (item.getKey().equals("metadata")) { - // json format in "metadata" - buffer.append(String.format(" \"%s\": %s,", item.getKey(), item.getValue())); - } else { - buffer.append(String.format(" \"%s\": \"%s\",", item.getKey(), item.getValue())); - } - hasValue = true; - } - if (hasValue) { - buffer.delete(buffer.length() - 1, buffer.length()); - } - buffer.append(" }"); - - return buffer.toString(); - } - - public static void main(String[] args) { - try{ - // entity without data value -// System.out.println("=== Test Entity without value stream ==="); - FileOutputStream out_none = new FileOutputStream(new File("output_none.txt")); - CdmiJsonInputStreamEntity entity_none = new CdmiJsonInputStreamEntity(null, 1000); - entity_none.addMetadata("author", "ywang"); - entity_none.addMetadata("tag", "cdmi"); - entity_none.writeTo(out_none); - - // entity with data value -// System.out.println("=== Test Entity with value stream from file ==="); - FileInputStream in_file = new FileInputStream(new File("in_file.txt")); - FileOutputStream out_file = new FileOutputStream(new File("out_file.txt")); - CdmiJsonInputStreamEntity entity_file = new CdmiJsonInputStreamEntity(in_file, 1024); - entity_file.addMetadata("author", "ywang"); - entity_file.addMetadata("tag", "cdmi"); - entity_file.writeTo(out_file); - - // entity with random input stream -// System.out.println("=== Test Entity with value stream from random generator ==="); -// RandomInputStream in_rnd = new RandomInputStream(100, new Random(23), true, false); -// FileOutputStream out_rnd = new FileOutputStream(new File("output_rnd.txt")); -// CdmiJsonInputStreamEntity entity_rnd = new CdmiJsonInputStreamEntity(in_rnd, 1024); -// entity_rnd.addMetadata("author", "ywang"); -// entity_rnd.addMetadata("tag", "cdmi"); -// entity_rnd.writeTo(out_rnd); - - }catch(Exception e) { - e.printStackTrace(); - } - } -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.client.cdmi.util; + +import java.io.File; +import java.io.FileInputStream; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.UnsupportedEncodingException; +import java.util.Map; +import java.util.TreeMap; + +import org.apache.http.Header; +import org.apache.http.entity.AbstractHttpEntity; +import org.apache.http.message.BasicHeader; + +/** + * This class encapsulates an input stream which can handle the json structure adopted by cdmi content type. + * + * @author ywang19 + * + */ +public class CdmiJsonInputStreamEntity extends AbstractHttpEntity +{ + private static final int BUFFER_SIZE = 2048; + private InputStream content; + private long length; // for value only in raw + private long transferLen; // for data actually transferred in entity + + private TreeMap data = new TreeMap() ; + private TreeMap metadata = new TreeMap(); + + public CdmiJsonInputStreamEntity() + { + this.content = null; + this.length = -1; + this.transferLen = -1; + } + + public CdmiJsonInputStreamEntity(InputStream instream, long length) + { + content = instream; + this.length = length; + this.transferLen = -1; + } + + public boolean isRepeatable() + { + if(content == null) + return true; + return content.markSupported(); + } + + public long getContentLength() + { + return transferLen; + } + + public long getTransferLength() + { + return transferLen; + } + + public InputStream getContent() + throws IOException + { + return content; + } + + public void addMetadata(String key, String value) { + metadata.put(key, value); + } + + public void addField(String key, String value) { + data.put(key, value); + } + + public void addContent(InputStream content, long length) { + this.content = content; + this.length = length; + } + + private String encodeEntity() { + data.put("metadata", encodeJson(metadata)); + + if(content != null) { + addField("mimetype", "text/plain"); + addField("valuetransferencoding", "utf-8"); + } + + return encodeJson(data); + } + + private long writePre(OutputStream outstream) throws UnsupportedEncodingException, IOException { + StringBuffer buffer = new StringBuffer(encodeEntity()); + if(length > 0) { + buffer.replace(buffer.length() - 1, buffer.length(), ","); + buffer.append(" \"value\": \""); + } + byte[] bytes = buffer.toString().getBytes("utf-8"); + + outstream.write(bytes); + + return bytes.length; + } + + private long writePost(OutputStream outstream) throws UnsupportedEncodingException, IOException { + if(length > 0) { + StringBuffer buffer = new StringBuffer("\" }"); + byte[] bytes = buffer.toString().getBytes("UTF-8"); + + outstream.write(bytes); + outstream.flush(); + + return bytes.length; + }else { + outstream.flush(); + + return length; + } + } + + public boolean isChunked() + { + return false; + } + + public Header getContentType() + { + return new BasicHeader("Content-Type", "application/cdmi-object"); + } + + /** + * Write value stream to output stream + * + * @param instream + * @param outstream + * @return + * @throws IOException + */ + private long writeContent(InputStream instream, OutputStream outstream) + { + if(instream == null) + return 0L; + + byte buffer[] = new byte[BUFFER_SIZE]; + int l; + long valueLen = 0; + + try{ + if(length < 0L) { + while((l = instream.read(buffer)) != -1) { + outstream.write(buffer, 0, l); + valueLen += l; + } + } else { + long remaining = length; + do + { + if(remaining <= 0L) + break; + l = instream.read(buffer, 0, (int)Math.min(BUFFER_SIZE, remaining)); + if(l == -1) + break; + outstream.write(buffer, 0, l); + valueLen += l; + remaining -= l; + } while(true); + } + }catch(IOException e) { + e.printStackTrace(); + } + + return valueLen; + } + + public void writeTo(OutputStream outstream) throws IOException + { + transferLen = 0; + if(outstream == null) + throw new IllegalArgumentException("Output stream may not be null"); + InputStream instream = content; + + try { + transferLen += writePre(outstream); + transferLen += writeContent(instream, outstream); + transferLen += writePost(outstream); + }finally { + if(instream != null) + instream.close(); + } + } + + public boolean isStreaming() + { + return true; + } + + /** + * @deprecated Method consumeContent is deprecated + */ + + public void consumeContent() + throws IOException + { + content.close(); + } + + /** + * Encode all items except "value" into json format. + * + * @param entries + * @return + */ + private static String encodeJson(Map entries) { + boolean hasValue = false; + StringBuffer buffer = new StringBuffer(); + buffer.append('{'); + for (Map.Entry item : entries.entrySet()) { + // filter out "value" + if (item.getKey().equals("value")) { + continue; + } + + if (item.getKey().equals("metadata")) { + // json format in "metadata" + buffer.append(String.format(" \"%s\": %s,", item.getKey(), item.getValue())); + } else { + buffer.append(String.format(" \"%s\": \"%s\",", item.getKey(), item.getValue())); + } + hasValue = true; + } + if (hasValue) { + buffer.delete(buffer.length() - 1, buffer.length()); + } + buffer.append(" }"); + + return buffer.toString(); + } + + public static void main(String[] args) { + try{ + // entity without data value +// System.out.println("=== Test Entity without value stream ==="); + FileOutputStream out_none = new FileOutputStream(new File("output_none.txt")); + CdmiJsonInputStreamEntity entity_none = new CdmiJsonInputStreamEntity(null, 1000); + entity_none.addMetadata("author", "ywang"); + entity_none.addMetadata("tag", "cdmi"); + entity_none.writeTo(out_none); + + // entity with data value +// System.out.println("=== Test Entity with value stream from file ==="); + FileInputStream in_file = new FileInputStream(new File("in_file.txt")); + FileOutputStream out_file = new FileOutputStream(new File("out_file.txt")); + CdmiJsonInputStreamEntity entity_file = new CdmiJsonInputStreamEntity(in_file, 1024); + entity_file.addMetadata("author", "ywang"); + entity_file.addMetadata("tag", "cdmi"); + entity_file.writeTo(out_file); + + // entity with random input stream +// System.out.println("=== Test Entity with value stream from random generator ==="); +// RandomInputStream in_rnd = new RandomInputStream(100, new Random(23), true, false); +// FileOutputStream out_rnd = new FileOutputStream(new File("output_rnd.txt")); +// CdmiJsonInputStreamEntity entity_rnd = new CdmiJsonInputStreamEntity(in_rnd, 1024); +// entity_rnd.addMetadata("author", "ywang"); +// entity_rnd.addMetadata("tag", "cdmi"); +// entity_rnd.writeTo(out_rnd); + + }catch(Exception e) { + e.printStackTrace(); + } + } +} diff --git a/dev/cosbench-config/META-INF/MANIFEST.MF b/dev/cosbench-config/META-INF/MANIFEST.MF index 5fe60eb9..29b5967c 100644 --- a/dev/cosbench-config/META-INF/MANIFEST.MF +++ b/dev/cosbench-config/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Config Bundle Bundle-SymbolicName: cosbench-config -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Export-Package: com.intel.cosbench.config, diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Auth.java b/dev/cosbench-config/src/com/intel/cosbench/config/Auth.java index fd63fce5..4e3e1ef5 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Auth.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Auth.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; @@ -21,8 +21,8 @@ /** * The model class mapping to "auth" in configuration xml with following form: - * - * + * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Config.java b/dev/cosbench-config/src/com/intel/cosbench/config/Config.java index 19d17bc4..8a40a726 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Config.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Config.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; /** * The interface of Configuration class. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/ConfigConstants.java b/dev/cosbench-config/src/com/intel/cosbench/config/ConfigConstants.java index 95a31f27..79608fdc 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/ConfigConstants.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/ConfigConstants.java @@ -1,7 +1,25 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.config; public interface ConfigConstants { - String DELIMITER=";"; - + String DELIMITER=";"; + } diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/ConfigException.java b/dev/cosbench-config/src/com/intel/cosbench/config/ConfigException.java index 7e01c1eb..7341f34f 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/ConfigException.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/ConfigException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,15 +12,15 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; /** * Configuration related exception. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Mission.java b/dev/cosbench-config/src/com/intel/cosbench/config/Mission.java index 88a706da..5c1906d4 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Mission.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Mission.java @@ -1,260 +1,260 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.config; - -import java.util.*; - -import org.apache.commons.lang.StringUtils; - -import com.intel.cosbench.config.common.ConfigUtils; -import com.intel.cosbench.config.common.KVConfigParser; - -/** - * The class encapsulates the mission delivering to driver. - * - * @author ywang19, qzheng7 - * - */ -public class Mission implements Iterable { - - private static final Auth DEFAULT_AUTH = new Auth("none"); - private static final Storage DEFAULT_STORAGE = new Storage("none"); - - private String name; - private int workers; - private int offset = 0; - private int interval = 5; - private String division = "none"; - private int runtime = 0; - private int rampup = 0; - private int rampdown = 0; - private int totalOps = 0; - private long totalBytes = 0; - private int totalWorkers; - private String config = ""; - private Auth auth = DEFAULT_AUTH; - private Storage storage = DEFAULT_STORAGE; - private List operations; - - - - public Mission() { - /* empty */ - } - - public String getName() { - return name; - } - - public void setName(String name) { - if (StringUtils.isEmpty(name)) - throw new ConfigException("mission name cannot be empty"); - this.name = name; - } - - public int getWorkers() { - return workers; - } - - public void setWorkers(int workers) { - if (workers < 0) - throw new ConfigException("illegal workers: " + workers); - if (workers == 0) - throw new ConfigException("must specify 'workers' for a mission"); - this.workers = workers; - } - - public int getOffset() { - return offset; - } - - public void setOffset(int offset) { - if (offset < 0) - throw new ConfigException("illegal offset: " + offset); - this.offset = offset; - } - - public int getInterval() { - return interval; - } - - public void setInterval(int interval) { - if (interval <= 0) - throw new ConfigException("illegal interval: " + interval); - this.interval = interval; - } - - public String getDivision() { - return division; - } - - public void setDivision(String division) { - if (StringUtils.isEmpty(division)) - throw new ConfigException("mission must have a default division"); - this.division = division; - } - - public int getRuntime() { - return runtime; - } - - public void setRuntime(int runtime) { - if (runtime < 0) - /* runtime can be set to zero */ - throw new ConfigException("illegal runtime: " + runtime); - this.runtime = runtime; - } - - public int getRampup() { - return rampup; - } - - public void setRampup(int rampup) { - if (rampup < 0) - /* ramp up time can be set to zero */ - throw new ConfigException("illegal ramp up time: " + rampup); - this.rampup = rampup; - } - - public int getRampdown() { - return rampdown; - } - - public void setRampdown(int rampdown) { - if (rampdown < 0) - /* ramp up time can be set to zero */ - throw new ConfigException("illegal ramp down time: " + rampdown); - this.rampdown = rampdown; - } - - public int getTotalOps() { - return totalOps; - } - - public void setTotalOps(int totalOps) { - if (totalOps < 0) - /* total operations can be set to zero */ - throw new ConfigException("illegal total ops: " + totalOps); - this.totalOps = totalOps; - } - - public long getTotalBytes() { - return totalBytes; - } - - public void setTotalBytes(long totalBytes) { - if (totalBytes < 0) - /* total bytes can be set to zero */ - throw new ConfigException("illegal total bytes: " + totalBytes); - this.totalBytes = totalBytes; - } - - public int getTotalWorkers() { - return totalWorkers; - } - - public void setTotalWorkers(int totalWorkers) { - if (totalWorkers < 0) - throw new ConfigException("illegal total workers: " + totalWorkers); - if (totalWorkers == 0) - throw new ConfigException( - "must specify 'totalWorkers' for a mission"); - this.totalWorkers = totalWorkers; - } - - public boolean hasHisto() { - if(config != null) { - return KVConfigParser.parse(config).getBoolean("histo", true); - } - - return true; - } - - public String getConfig() { - return config; - } - - public void setConfig(String config) { - /* configuration might be empty */ - this.config = config; - } - - public Auth getAuth() { - return auth; - } - - public void setAuth(Auth auth) { - if (auth == null) - throw new ConfigException("a mission must have its auth"); - this.auth = auth; - } - - public Storage getStorage() { - return storage; - } - - public void setStorage(Storage storage) { - if (storage == null) - throw new ConfigException("a mission must have its storge"); - this.storage = storage; - } - - public List getOperations() { - return operations; - } - - public void setOperations(List operations) { - if (operations == null || operations.isEmpty()) - throw new ConfigException("a mission must have opertations"); - for(Operation op: operations) { - op.setConfig(ConfigUtils.inherit(op.getConfig(), this.config)); - } - this.operations = operations; - } - - @Override - public Iterator iterator() { - return operations.iterator(); - } - - public void validate() { - setName(getName()); - setWorkers(getWorkers()); - setTotalWorkers(getTotalWorkers()); - if (totalWorkers < workers || offset + workers > totalWorkers) - throw new ConfigException( - "conflicting workers, totalWorkers, and offset"); - if (runtime == 0 && totalOps == 0 && totalBytes == 0) - throw new ConfigException( - "no mission limits detectd, either runtime, total ops or total bytes has been set"); - auth.validate(); - storage.validate(); - setOperations(getOperations()); - for (Operation op : operations) - if (op.getDivision() == null) - op.setDivision(division); - for (Operation op : operations) - op.validate(); - int sum = 0; - for (Operation op : operations) - sum += op.getRatio(); - if (sum != 100) - throw new ConfigException("op ratio should sum to 100"); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.config; + +import java.util.*; + +import org.apache.commons.lang.StringUtils; + +import com.intel.cosbench.config.common.ConfigUtils; +import com.intel.cosbench.config.common.KVConfigParser; + +/** + * The class encapsulates the mission delivering to driver. + * + * @author ywang19, qzheng7 + * + */ +public class Mission implements Iterable { + + private static final Auth DEFAULT_AUTH = new Auth("none"); + private static final Storage DEFAULT_STORAGE = new Storage("none"); + + private String name; + private int workers; + private int offset = 0; + private int interval = 5; + private String division = "none"; + private int runtime = 0; + private int rampup = 0; + private int rampdown = 0; + private int totalOps = 0; + private long totalBytes = 0; + private int totalWorkers; + private String config = ""; + private Auth auth = DEFAULT_AUTH; + private Storage storage = DEFAULT_STORAGE; + private List operations; + + + + public Mission() { + /* empty */ + } + + public String getName() { + return name; + } + + public void setName(String name) { + if (StringUtils.isEmpty(name)) + throw new ConfigException("mission name cannot be empty"); + this.name = name; + } + + public int getWorkers() { + return workers; + } + + public void setWorkers(int workers) { + if (workers < 0) + throw new ConfigException("illegal workers: " + workers); + if (workers == 0) + throw new ConfigException("must specify 'workers' for a mission"); + this.workers = workers; + } + + public int getOffset() { + return offset; + } + + public void setOffset(int offset) { + if (offset < 0) + throw new ConfigException("illegal offset: " + offset); + this.offset = offset; + } + + public int getInterval() { + return interval; + } + + public void setInterval(int interval) { + if (interval <= 0) + throw new ConfigException("illegal interval: " + interval); + this.interval = interval; + } + + public String getDivision() { + return division; + } + + public void setDivision(String division) { + if (StringUtils.isEmpty(division)) + throw new ConfigException("mission must have a default division"); + this.division = division; + } + + public int getRuntime() { + return runtime; + } + + public void setRuntime(int runtime) { + if (runtime < 0) + /* runtime can be set to zero */ + throw new ConfigException("illegal runtime: " + runtime); + this.runtime = runtime; + } + + public int getRampup() { + return rampup; + } + + public void setRampup(int rampup) { + if (rampup < 0) + /* ramp up time can be set to zero */ + throw new ConfigException("illegal ramp up time: " + rampup); + this.rampup = rampup; + } + + public int getRampdown() { + return rampdown; + } + + public void setRampdown(int rampdown) { + if (rampdown < 0) + /* ramp up time can be set to zero */ + throw new ConfigException("illegal ramp down time: " + rampdown); + this.rampdown = rampdown; + } + + public int getTotalOps() { + return totalOps; + } + + public void setTotalOps(int totalOps) { + if (totalOps < 0) + /* total operations can be set to zero */ + throw new ConfigException("illegal total ops: " + totalOps); + this.totalOps = totalOps; + } + + public long getTotalBytes() { + return totalBytes; + } + + public void setTotalBytes(long totalBytes) { + if (totalBytes < 0) + /* total bytes can be set to zero */ + throw new ConfigException("illegal total bytes: " + totalBytes); + this.totalBytes = totalBytes; + } + + public int getTotalWorkers() { + return totalWorkers; + } + + public void setTotalWorkers(int totalWorkers) { + if (totalWorkers < 0) + throw new ConfigException("illegal total workers: " + totalWorkers); + if (totalWorkers == 0) + throw new ConfigException( + "must specify 'totalWorkers' for a mission"); + this.totalWorkers = totalWorkers; + } + + public boolean hasHisto() { + if(config != null) { + return KVConfigParser.parse(config).getBoolean("histo", true); + } + + return true; + } + + public String getConfig() { + return config; + } + + public void setConfig(String config) { + /* configuration might be empty */ + this.config = config; + } + + public Auth getAuth() { + return auth; + } + + public void setAuth(Auth auth) { + if (auth == null) + throw new ConfigException("a mission must have its auth"); + this.auth = auth; + } + + public Storage getStorage() { + return storage; + } + + public void setStorage(Storage storage) { + if (storage == null) + throw new ConfigException("a mission must have its storge"); + this.storage = storage; + } + + public List getOperations() { + return operations; + } + + public void setOperations(List operations) { + if (operations == null || operations.isEmpty()) + throw new ConfigException("a mission must have opertations"); + for(Operation op: operations) { + op.setConfig(ConfigUtils.inherit(op.getConfig(), this.config)); + } + this.operations = operations; + } + + @Override + public Iterator iterator() { + return operations.iterator(); + } + + public void validate() { + setName(getName()); + setWorkers(getWorkers()); + setTotalWorkers(getTotalWorkers()); + if (totalWorkers < workers || offset + workers > totalWorkers) + throw new ConfigException( + "conflicting workers, totalWorkers, and offset"); + if (runtime == 0 && totalOps == 0 && totalBytes == 0) + throw new ConfigException( + "no mission limits detectd, either runtime, total ops or total bytes has been set"); + auth.validate(); + storage.validate(); + setOperations(getOperations()); + for (Operation op : operations) + if (op.getDivision() == null) + op.setDivision(division); + for (Operation op : operations) + op.validate(); + int sum = 0; + for (Operation op : operations) + sum += op.getRatio(); + if (sum != 100) + throw new ConfigException("op ratio should sum to 100"); + } + +} diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/MissionResolver.java b/dev/cosbench-config/src/com/intel/cosbench/config/MissionResolver.java index c73ef73d..221d3a0b 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/MissionResolver.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/MissionResolver.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/MissionWriter.java b/dev/cosbench-config/src/com/intel/cosbench/config/MissionWriter.java index 99909dbb..dbfd0777 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/MissionWriter.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/MissionWriter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Operation.java b/dev/cosbench-config/src/com/intel/cosbench/config/Operation.java index f731866d..7e2bcaec 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Operation.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Operation.java @@ -1,100 +1,100 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.config; - -import org.apache.commons.lang.StringUtils; - -/** - * The model class mapping to "operation" in configuration xml with following form: - * - * - * @author ywang19, qzheng7 - * - */ -public class Operation { - - private String type; - private int ratio = 100; - private String division; - private String config = ""; - private String id = "none"; /* will be inited in workloadProcessor.initStageOpId() */ - - public Operation() { - /* empty */ - } - - public Operation(String type) { - setType(type); - } - - public String getType() { - return type; - } - - public void setType(String type) { - if (StringUtils.isEmpty(type)) - throw new ConfigException("operation type cannot be empty"); - this.type = type; - } - - public int getRatio() { - return ratio; - } - - public void setRatio(int ratio) { - if (ratio < 0 || ratio > 100) - /* ratio can be set to zero */ - throw new ConfigException("illegal operation ratio: " + ratio); - this.ratio = ratio; - } - - public String getDivision() { - return division; - } - - public void setDivision(String division) { - if (StringUtils.isEmpty(division)) - throw new ConfigException("oepration must have its division"); - this.division = division; - } - - public String getConfig() { - return config; - } - - public void setConfig(String config) { - /* configuration might be empty */ - this.config = config; - } - - - public String getId() { - return id; - } - - public void setId(String id) { - this.id = id; - } - - - public void validate() { - setType(getType()); - setDivision(getDivision()); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.config; + +import org.apache.commons.lang.StringUtils; + +/** + * The model class mapping to "operation" in configuration xml with following form: + * + * + * @author ywang19, qzheng7 + * + */ +public class Operation { + + private String type; + private int ratio = 100; + private String division; + private String config = ""; + private String id = "none"; /* will be inited in workloadProcessor.initStageOpId() */ + + public Operation() { + /* empty */ + } + + public Operation(String type) { + setType(type); + } + + public String getType() { + return type; + } + + public void setType(String type) { + if (StringUtils.isEmpty(type)) + throw new ConfigException("operation type cannot be empty"); + this.type = type; + } + + public int getRatio() { + return ratio; + } + + public void setRatio(int ratio) { + if (ratio < 0 || ratio > 100) + /* ratio can be set to zero */ + throw new ConfigException("illegal operation ratio: " + ratio); + this.ratio = ratio; + } + + public String getDivision() { + return division; + } + + public void setDivision(String division) { + if (StringUtils.isEmpty(division)) + throw new ConfigException("oepration must have its division"); + this.division = division; + } + + public String getConfig() { + return config; + } + + public void setConfig(String config) { + /* configuration might be empty */ + this.config = config; + } + + + public String getId() { + return id; + } + + public void setId(String id) { + this.id = id; + } + + + public void validate() { + setType(getType()); + setDivision(getDivision()); + } + +} diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Stage.java b/dev/cosbench-config/src/com/intel/cosbench/config/Stage.java index 03ca9fa1..0eec6195 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Stage.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Stage.java @@ -1,180 +1,180 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.config; - -import java.util.*; - -import org.apache.commons.lang.StringUtils; - -import com.intel.cosbench.config.common.ConfigUtils; -import com.intel.cosbench.log.LogFactory; -import com.intel.cosbench.log.Logger; - -/** - * The model class mapping to "workstage" in configuration xml with following form: - * - * - * @author ywang19, qzheng7 - * - */ -public class Stage implements Iterable { - - private String name; - private int closuredelay; - private String trigger=null; - private String config = ""; - private Auth auth; - private Storage storage; - private List works; - - public Stage() { - /* empty */ - } - - public Stage(String name) { - setName(name); - } - - public String getName() { - return name; - } - - public void setName(String name) { - if (StringUtils.isEmpty(name)) - throw new ConfigException("stage name cannot be empty"); - if (StringUtils.containsAny(name, ConfigConstants.DELIMITER)) - throw new ConfigException("stage name cannot contain delimiter '" + ConfigConstants.DELIMITER + "'"); - this.name = name; - } - - public int getClosuredelay() { - return closuredelay; - } - - public String getTrigger() { - return trigger; - } - - public void setTrigger(String trigger) { - this.trigger = trigger; - } - - public String getConfig() { - return config; - } - - public void setConfig(String config) { - this.config = config; - } - - public void setClosuredelay(int closuredelay) { - if (closuredelay < 0) - throw new ConfigException("closure delay cannot be negative"); - this.closuredelay = closuredelay; - } - - public Auth getAuth() { - return auth; - } - - public void setAuth(Auth auth) { - if (auth == null) - throw new ConfigException("a stage must have a default auth"); - this.auth = auth; - } - - public Storage getStorage() { - return storage; - } - - public void setStorage(Storage storage) { - if (storage == null) - throw new ConfigException("a stage must have a default storage"); -// if(!name.equals("init") && !name.equals("dispose")) -// this.storage=removeNSROOTConfig(storage); -// else - this.storage = storage; - } - - // method for removing nsroot config from prepare, normal and cleanup stages -// private Storage removeNSROOTConfig(Storage storage) { -// if(storage.getConfig() == null) -// return storage; -// if (!storage.getConfig().contains("nsroot")) -// return storage; -// else { -// Storage newStorage = new Storage(); -// String configParams[] = storage.getConfig().split(";"); -// StringBuffer newConfig = new StringBuffer(""); -// for (String configParam : configParams) { -// if (!configParam.toLowerCase().contains("nsroot")) -// newConfig.append(configParam + ";"); -// } -// newConfig.deleteCharAt(newConfig.length() - 1); -// newStorage.setType(storage.getType()); -// newStorage.setConfig(newConfig.toString()); -// return newStorage; -// } -// } - - public List getWorks() { - return works; - } - - public void setWorks(List works) { - if (works == null || works.isEmpty()) - throw new ConfigException("stage must have works"); - for(Work work: works) { - work.setConfig(ConfigUtils.inherit(work.getConfig(), this.config)); - Logger logger = LogFactory.getSystemLogger(); - logger.debug("stage config: "+this.config+ "work inherit result: "+ ConfigUtils.inherit(work.getConfig(), this.config)); - - } - this.works = works; - } - - public void addWork(Work work) { - if (work == null || work.getName().isEmpty()) - throw new ConfigException("can't add one empty work"); - if (works == null) - works = new ArrayList(); - work.setConfig(ConfigUtils.inherit(work.getConfig(), this.config)); - works.add(work); - } - - @Override - public Iterator iterator() { - return works.iterator(); - } - - public void validate() { - setName(getName()); - setAuth(getAuth()); - setStorage(getStorage()); - setWorks(getWorks()); - for (Work work : works) - if (work.getAuth() == null) - work.setAuth(auth); - for (Work work : works) - if (work.getStorage() == null) - work.setStorage(storage); - for (Work work : works) - work.validate(); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.config; + +import java.util.*; + +import org.apache.commons.lang.StringUtils; + +import com.intel.cosbench.config.common.ConfigUtils; +import com.intel.cosbench.log.LogFactory; +import com.intel.cosbench.log.Logger; + +/** + * The model class mapping to "workstage" in configuration xml with following form: + * + * + * @author ywang19, qzheng7 + * + */ +public class Stage implements Iterable { + + private String name; + private int closuredelay; + private String trigger=null; + private String config = ""; + private Auth auth; + private Storage storage; + private List works; + + public Stage() { + /* empty */ + } + + public Stage(String name) { + setName(name); + } + + public String getName() { + return name; + } + + public void setName(String name) { + if (StringUtils.isEmpty(name)) + throw new ConfigException("stage name cannot be empty"); + if (StringUtils.containsAny(name, ConfigConstants.DELIMITER)) + throw new ConfigException("stage name cannot contain delimiter '" + ConfigConstants.DELIMITER + "'"); + this.name = name; + } + + public int getClosuredelay() { + return closuredelay; + } + + public String getTrigger() { + return trigger; + } + + public void setTrigger(String trigger) { + this.trigger = trigger; + } + + public String getConfig() { + return config; + } + + public void setConfig(String config) { + this.config = config; + } + + public void setClosuredelay(int closuredelay) { + if (closuredelay < 0) + throw new ConfigException("closure delay cannot be negative"); + this.closuredelay = closuredelay; + } + + public Auth getAuth() { + return auth; + } + + public void setAuth(Auth auth) { + if (auth == null) + throw new ConfigException("a stage must have a default auth"); + this.auth = auth; + } + + public Storage getStorage() { + return storage; + } + + public void setStorage(Storage storage) { + if (storage == null) + throw new ConfigException("a stage must have a default storage"); +// if(!name.equals("init") && !name.equals("dispose")) +// this.storage=removeNSROOTConfig(storage); +// else + this.storage = storage; + } + + // method for removing nsroot config from prepare, normal and cleanup stages +// private Storage removeNSROOTConfig(Storage storage) { +// if(storage.getConfig() == null) +// return storage; +// if (!storage.getConfig().contains("nsroot")) +// return storage; +// else { +// Storage newStorage = new Storage(); +// String configParams[] = storage.getConfig().split(";"); +// StringBuffer newConfig = new StringBuffer(""); +// for (String configParam : configParams) { +// if (!configParam.toLowerCase().contains("nsroot")) +// newConfig.append(configParam + ";"); +// } +// newConfig.deleteCharAt(newConfig.length() - 1); +// newStorage.setType(storage.getType()); +// newStorage.setConfig(newConfig.toString()); +// return newStorage; +// } +// } + + public List getWorks() { + return works; + } + + public void setWorks(List works) { + if (works == null || works.isEmpty()) + throw new ConfigException("stage must have works"); + for(Work work: works) { + work.setConfig(ConfigUtils.inherit(work.getConfig(), this.config)); + Logger logger = LogFactory.getSystemLogger(); + logger.debug("stage config: "+this.config+ "work inherit result: "+ ConfigUtils.inherit(work.getConfig(), this.config)); + + } + this.works = works; + } + + public void addWork(Work work) { + if (work == null || work.getName().isEmpty()) + throw new ConfigException("can't add one empty work"); + if (works == null) + works = new ArrayList(); + work.setConfig(ConfigUtils.inherit(work.getConfig(), this.config)); + works.add(work); + } + + @Override + public Iterator iterator() { + return works.iterator(); + } + + public void validate() { + setName(getName()); + setAuth(getAuth()); + setStorage(getStorage()); + setWorks(getWorks()); + for (Work work : works) + if (work.getAuth() == null) + work.setAuth(auth); + for (Work work : works) + if (work.getStorage() == null) + work.setStorage(storage); + for (Work work : works) + work.validate(); + } + +} diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Storage.java b/dev/cosbench-config/src/com/intel/cosbench/config/Storage.java index c3985839..f6bdce20 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Storage.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Storage.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; @@ -21,8 +21,8 @@ /** * The model class mapping to "storage" in configuration xml with following form: - * - * + * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Work.java b/dev/cosbench-config/src/com/intel/cosbench/config/Work.java index 9622f049..21e30300 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Work.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Work.java @@ -1,393 +1,393 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.config; - -import java.util.*; - -import org.apache.commons.lang.StringUtils; - -import com.intel.cosbench.config.common.ConfigUtils; - -/** - * The model class mapping to "work" in configuration xml with following form: - * - * - * @author ywang19, qzheng7 - * - */ -public class Work implements Iterable { - - private String name; - private String type = "normal"; - private int workers; - private int interval = 5; - private String division = "none"; - private int runtime = 0; - private int rampup = 0; - private int rampdown = 0; - private int afr = -1; /* acceptable failure ratio, the unit is samples per one million, - * default is 200000 for normal work, and 0 for init/prepare/cleanup/dispose/delay work */ - private int totalOps = 0; - private long totalBytes = 0; - private String driver; - private String config = ""; - private Auth auth; - private Storage storage; - private List operations; - - public Work() { - /* empty */ - } - - public Work(String name) { - setName(name); - } - - public Work(String name, String type) { - setName(name); - setType(type); - } - - public String getName() { - return name; - } - - public void setName(String name) { - if (StringUtils.isEmpty(name)) - throw new ConfigException("work name cannot be empty"); - this.name = name; - } - - public String getType() { - return type; - } - - public void setType(String type) { - if (StringUtils.isEmpty(type)) - throw new ConfigException("work type cannot be empty"); - this.type = type; - } - - public int getWorkers() { - return workers; - } - - public void setWorkers(int workers) { - if (workers < 0) - throw new ConfigException("illegal workers: " + workers); - if (workers == 0) - throw new ConfigException("must specify 'workers' for a work"); - this.workers = workers; - } - - public int getInterval() { - return interval; - } - - public void setInterval(int interval) { - if (interval <= 0) - throw new ConfigException("illegal interval: " + interval); - this.interval = interval; - } - - public String getDivision() { - return division; - } - - public void setDivision(String division) { - if (StringUtils.isEmpty(division)) - throw new ConfigException("work must have a default division"); - this.division = division; - } - - public int getRuntime() { - return runtime; - } - - public void setRuntime(int runtime) { - if (runtime < 0) - /* runtime can be set to zero */ - throw new ConfigException("illegal runtime: " + runtime); - this.runtime = runtime; - } - - public int getRampup() { - return rampup; - } - - public void setRampup(int rampup) { - if (rampup < 0) - /* ramp up time can be set to zero */ - throw new ConfigException("illegal ramp up time: " + rampup); - this.rampup = rampup; - } - - public int getRampdown() { - return rampdown; - } - - public void setRampdown(int rampdown) { - if (rampdown < 0) - /* ramp up time can be set to zero */ - throw new ConfigException("illegal ramp down time: " + rampdown); - this.rampdown = rampdown; - } - - public int getTotalOps() { - return totalOps; - } - - public void setTotalOps(int totalOps) { - if (totalOps < 0) - /* total operations can be set to zero */ - throw new ConfigException("illegal total ops: " + totalOps); - this.totalOps = totalOps; - } - - public long getTotalBytes() { - return totalBytes; - } - - public void setTotalBytes(long totalBytes) { - if (totalBytes < 0) - /* total bytes can be set to zero */ - throw new ConfigException("illegal total bytes: " + totalBytes); - this.totalBytes = totalBytes; - } - - public String getDriver() { - return driver; - } - - public void setDriver(String driver) { - /* driver might be empty */ - this.driver = driver; - } - - public String getConfig() { - return config; - } - - public void setConfig(String config) { - /* configuration might be empty */ - this.config = config; - } - - public Auth getAuth() { - return auth; - } - - public void setAuth(Auth auth) { - if (auth == null) - throw new ConfigException("a work must have its auth"); - this.auth = auth; - } - - public Storage getStorage() { - return storage; - } - - public void setStorage(Storage storage) { - if (storage == null) - throw new ConfigException("a work must have its storge"); - this.storage = storage; - } - - public int getAfr() { - return afr; - } - - public void setAfr(int afr) { - if (afr > 1000000 || afr < 0) - throw new ConfigException("afr should be at 0 to 1000000 range"); - this.afr = afr; - } - - public List getOperationIDs() { - List opIds = new ArrayList(); - for (Operation operation : operations) { - opIds.add(operation.getId()); - } - return opIds; - } - - - public List getOperations() { - return operations; - } - - public void setOperations(List operations) { - if (operations == null || operations.isEmpty()) - throw new ConfigException("a work must have opertations"); - for(Operation op: operations) { - op.setConfig(ConfigUtils.inherit(op.getConfig(), this.config)); - } - this.operations = operations; - } - - public void addOperation(Operation op) { - if (op == null || op.getType().isEmpty()) - throw new ConfigException("a operation must have type"); - if (operations == null) - operations = new ArrayList(); - op.setConfig(ConfigUtils.inherit(op.getConfig(), this.config)); - operations.add(op); - } - - @Override - public Iterator iterator() { - return operations.iterator(); - } - - private void toPrepareWork() { - if (name == null) - name = "prepare"; - setDivision("object"); - setRuntime(0); - setDefaultAfr(0); - setTotalBytes(0); - setTotalOps(getWorkers()); - Operation op = new Operation(); - op.setType("prepare"); - op.setRatio(100); - Object[] cfgs = null; - if (config.indexOf("createContainer=") < 0) - cfgs = new Object[] { "createContainer=false", config }; - else - cfgs = new Object[] { config }; - op.setConfig(StringUtils.join(cfgs, ';')); - setOperations(Collections.singletonList(op)); - } - - private void toCleanupWork() { - if (name == null) - name = "cleanup"; - setDivision("object"); - setRuntime(0); - setDefaultAfr(0); - setTotalBytes(0); - setTotalOps(getWorkers()); - Operation op = new Operation(); - op.setType("cleanup"); - op.setRatio(100); - Object[] cfgs = null; - if (config.indexOf("deleteContainer=") < 0) - cfgs = new Object[] { "deleteContainer=false", config }; - else - cfgs = new Object[] { config }; - op.setConfig(StringUtils.join(cfgs, ';')); - setOperations(Collections.singletonList(op)); - } - - private void toInitWork() { - if (name == null) - name = "init"; - setDivision("container"); - setRuntime(0); - setDefaultAfr(0); - setTotalBytes(0); - setTotalOps(getWorkers()); - Operation op = new Operation(); - op.setType("init"); - op.setRatio(100); - Object[] cfgs = new Object[] { "objects=r(0,0);sizes=c(0)B", config }; - op.setConfig(StringUtils.join(cfgs, ';')); - setOperations(Collections.singletonList(op)); - } - - private void toDisposeWork() { - if (name == null) - name = "dispose"; - setDivision("container"); - setRuntime(0); - setDefaultAfr(0); - setTotalBytes(0); - setTotalOps(getWorkers()); - Operation op = new Operation(); - op.setType("dispose"); - op.setRatio(100); - Object[] cfgs = new Object[] { "objects=r(0,0);sizes=c(0)B", config }; - op.setConfig(StringUtils.join(cfgs, ';')); - setOperations(Collections.singletonList(op)); - } - - public void toDelayWork() { - if (name == null) - name = "delay"; - setDivision("none"); - setRuntime(0); - setDefaultAfr(0); - setTotalBytes(0); - setWorkers(1); - setTotalOps(getWorkers()); - Operation op = new Operation(); - op.setType("delay"); - op.setRatio(100); - op.setConfig(""); - setOperations(Collections.singletonList(op)); - } - - private void setDefaultAfr(int def) { - if (afr < 0) - setAfr(def); - } - - public void validate() { - if (type.equals("prepare")) - toPrepareWork(); - else if (type.equals("cleanup")) - toCleanupWork(); - else if (type.equals("init")) - toInitWork(); - else if (type.equals("dispose")) - toDisposeWork(); - else if (type.equals("delay")) - toDelayWork(); - else - setDefaultAfr(200000); - setName(getName()); - setWorkers(getWorkers()); - if (runtime == 0 && totalOps == 0 && totalBytes == 0) - throw new ConfigException( - "no work limits detectd, either runtime, total ops or total bytes"); - setAuth(getAuth()); - auth.validate(); - setStorage(getStorage()); - storage.validate(); - List tempOpList = new ArrayList(); - for (Operation op: operations) { - if(op.getRatio() > 0) { - tempOpList.add(op); - } - } - operations = tempOpList; - setOperations(getOperations()); - for (Operation op : operations) - if (op.getDivision() == null) - op.setDivision(division); - for (Operation op : operations) - op.validate(); - int sum = 0; - for (Operation op : operations) - sum += op.getRatio(); - if (sum != 100) - throw new ConfigException("op ratio should sum to 100"); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.config; + +import java.util.*; + +import org.apache.commons.lang.StringUtils; + +import com.intel.cosbench.config.common.ConfigUtils; + +/** + * The model class mapping to "work" in configuration xml with following form: + * + * + * @author ywang19, qzheng7 + * + */ +public class Work implements Iterable { + + private String name; + private String type = "normal"; + private int workers; + private int interval = 5; + private String division = "none"; + private int runtime = 0; + private int rampup = 0; + private int rampdown = 0; + private int afr = -1; /* acceptable failure ratio, the unit is samples per one million, + * default is 200000 for normal work, and 0 for init/prepare/cleanup/dispose/delay work */ + private int totalOps = 0; + private long totalBytes = 0; + private String driver; + private String config = ""; + private Auth auth; + private Storage storage; + private List operations; + + public Work() { + /* empty */ + } + + public Work(String name) { + setName(name); + } + + public Work(String name, String type) { + setName(name); + setType(type); + } + + public String getName() { + return name; + } + + public void setName(String name) { + if (StringUtils.isEmpty(name)) + throw new ConfigException("work name cannot be empty"); + this.name = name; + } + + public String getType() { + return type; + } + + public void setType(String type) { + if (StringUtils.isEmpty(type)) + throw new ConfigException("work type cannot be empty"); + this.type = type; + } + + public int getWorkers() { + return workers; + } + + public void setWorkers(int workers) { + if (workers < 0) + throw new ConfigException("illegal workers: " + workers); + if (workers == 0) + throw new ConfigException("must specify 'workers' for a work"); + this.workers = workers; + } + + public int getInterval() { + return interval; + } + + public void setInterval(int interval) { + if (interval <= 0) + throw new ConfigException("illegal interval: " + interval); + this.interval = interval; + } + + public String getDivision() { + return division; + } + + public void setDivision(String division) { + if (StringUtils.isEmpty(division)) + throw new ConfigException("work must have a default division"); + this.division = division; + } + + public int getRuntime() { + return runtime; + } + + public void setRuntime(int runtime) { + if (runtime < 0) + /* runtime can be set to zero */ + throw new ConfigException("illegal runtime: " + runtime); + this.runtime = runtime; + } + + public int getRampup() { + return rampup; + } + + public void setRampup(int rampup) { + if (rampup < 0) + /* ramp up time can be set to zero */ + throw new ConfigException("illegal ramp up time: " + rampup); + this.rampup = rampup; + } + + public int getRampdown() { + return rampdown; + } + + public void setRampdown(int rampdown) { + if (rampdown < 0) + /* ramp up time can be set to zero */ + throw new ConfigException("illegal ramp down time: " + rampdown); + this.rampdown = rampdown; + } + + public int getTotalOps() { + return totalOps; + } + + public void setTotalOps(int totalOps) { + if (totalOps < 0) + /* total operations can be set to zero */ + throw new ConfigException("illegal total ops: " + totalOps); + this.totalOps = totalOps; + } + + public long getTotalBytes() { + return totalBytes; + } + + public void setTotalBytes(long totalBytes) { + if (totalBytes < 0) + /* total bytes can be set to zero */ + throw new ConfigException("illegal total bytes: " + totalBytes); + this.totalBytes = totalBytes; + } + + public String getDriver() { + return driver; + } + + public void setDriver(String driver) { + /* driver might be empty */ + this.driver = driver; + } + + public String getConfig() { + return config; + } + + public void setConfig(String config) { + /* configuration might be empty */ + this.config = config; + } + + public Auth getAuth() { + return auth; + } + + public void setAuth(Auth auth) { + if (auth == null) + throw new ConfigException("a work must have its auth"); + this.auth = auth; + } + + public Storage getStorage() { + return storage; + } + + public void setStorage(Storage storage) { + if (storage == null) + throw new ConfigException("a work must have its storge"); + this.storage = storage; + } + + public int getAfr() { + return afr; + } + + public void setAfr(int afr) { + if (afr > 1000000 || afr < 0) + throw new ConfigException("afr should be at 0 to 1000000 range"); + this.afr = afr; + } + + public List getOperationIDs() { + List opIds = new ArrayList(); + for (Operation operation : operations) { + opIds.add(operation.getId()); + } + return opIds; + } + + + public List getOperations() { + return operations; + } + + public void setOperations(List operations) { + if (operations == null || operations.isEmpty()) + throw new ConfigException("a work must have opertations"); + for(Operation op: operations) { + op.setConfig(ConfigUtils.inherit(op.getConfig(), this.config)); + } + this.operations = operations; + } + + public void addOperation(Operation op) { + if (op == null || op.getType().isEmpty()) + throw new ConfigException("a operation must have type"); + if (operations == null) + operations = new ArrayList(); + op.setConfig(ConfigUtils.inherit(op.getConfig(), this.config)); + operations.add(op); + } + + @Override + public Iterator iterator() { + return operations.iterator(); + } + + private void toPrepareWork() { + if (name == null) + name = "prepare"; + setDivision("object"); + setRuntime(0); + setDefaultAfr(0); + setTotalBytes(0); + setTotalOps(getWorkers()); + Operation op = new Operation(); + op.setType("prepare"); + op.setRatio(100); + Object[] cfgs = null; + if (config.indexOf("createContainer=") < 0) + cfgs = new Object[] { "createContainer=false", config }; + else + cfgs = new Object[] { config }; + op.setConfig(StringUtils.join(cfgs, ';')); + setOperations(Collections.singletonList(op)); + } + + private void toCleanupWork() { + if (name == null) + name = "cleanup"; + setDivision("object"); + setRuntime(0); + setDefaultAfr(0); + setTotalBytes(0); + setTotalOps(getWorkers()); + Operation op = new Operation(); + op.setType("cleanup"); + op.setRatio(100); + Object[] cfgs = null; + if (config.indexOf("deleteContainer=") < 0) + cfgs = new Object[] { "deleteContainer=false", config }; + else + cfgs = new Object[] { config }; + op.setConfig(StringUtils.join(cfgs, ';')); + setOperations(Collections.singletonList(op)); + } + + private void toInitWork() { + if (name == null) + name = "init"; + setDivision("container"); + setRuntime(0); + setDefaultAfr(0); + setTotalBytes(0); + setTotalOps(getWorkers()); + Operation op = new Operation(); + op.setType("init"); + op.setRatio(100); + Object[] cfgs = new Object[] { "objects=r(0,0);sizes=c(0)B", config }; + op.setConfig(StringUtils.join(cfgs, ';')); + setOperations(Collections.singletonList(op)); + } + + private void toDisposeWork() { + if (name == null) + name = "dispose"; + setDivision("container"); + setRuntime(0); + setDefaultAfr(0); + setTotalBytes(0); + setTotalOps(getWorkers()); + Operation op = new Operation(); + op.setType("dispose"); + op.setRatio(100); + Object[] cfgs = new Object[] { "objects=r(0,0);sizes=c(0)B", config }; + op.setConfig(StringUtils.join(cfgs, ';')); + setOperations(Collections.singletonList(op)); + } + + public void toDelayWork() { + if (name == null) + name = "delay"; + setDivision("none"); + setRuntime(0); + setDefaultAfr(0); + setTotalBytes(0); + setWorkers(1); + setTotalOps(getWorkers()); + Operation op = new Operation(); + op.setType("delay"); + op.setRatio(100); + op.setConfig(""); + setOperations(Collections.singletonList(op)); + } + + private void setDefaultAfr(int def) { + if (afr < 0) + setAfr(def); + } + + public void validate() { + if (type.equals("prepare")) + toPrepareWork(); + else if (type.equals("cleanup")) + toCleanupWork(); + else if (type.equals("init")) + toInitWork(); + else if (type.equals("dispose")) + toDisposeWork(); + else if (type.equals("delay")) + toDelayWork(); + else + setDefaultAfr(200000); + setName(getName()); + setWorkers(getWorkers()); + if (runtime == 0 && totalOps == 0 && totalBytes == 0) + throw new ConfigException( + "no work limits detectd, either runtime, total ops or total bytes"); + setAuth(getAuth()); + auth.validate(); + setStorage(getStorage()); + storage.validate(); + List tempOpList = new ArrayList(); + for (Operation op: operations) { + if(op.getRatio() > 0) { + tempOpList.add(op); + } + } + operations = tempOpList; + setOperations(getOperations()); + for (Operation op : operations) + if (op.getDivision() == null) + op.setDivision(division); + for (Operation op : operations) + op.validate(); + int sum = 0; + for (Operation op : operations) + sum += op.getRatio(); + if (sum != 100) + throw new ConfigException("op ratio should sum to 100"); + } + +} diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Workflow.java b/dev/cosbench-config/src/com/intel/cosbench/config/Workflow.java index 81ff9a66..9b1028b1 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Workflow.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Workflow.java @@ -1,83 +1,86 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.config; - -import java.util.*; - -import com.intel.cosbench.config.common.ConfigUtils; -import com.intel.cosbench.log.LogFactory; -import com.intel.cosbench.log.Logger; - -/** - * The model class mapping to "workflow" in configuration xml with following form: - * - * - * @author ywang19, qzheng7 - * - */ -public class Workflow implements Iterable { - - private List stages; - private String config = ""; - - public Workflow() { - /* empty */ - } - - public String getConfig() { - return config; - } - - public void setConfig(String config) { - this.config = config; - } - - public List getStages() { - return stages; - } - - public void setStages(List stages) { - if (stages == null || stages.isEmpty()) - throw new ConfigException("workflow must have stages"); - for(Stage stage: stages) { - stage.setConfig(ConfigUtils.inherit(stage.getConfig(), this.config)); - } - this.stages = stages; - } - - public void addStage(Stage stage) { - if (stage == null || stage.getName().isEmpty()) - throw new ConfigException("can't add one empty stage"); - if (stages == null) - stages = new ArrayList(); - stage.setConfig(ConfigUtils.inherit(stage.getConfig(), this.config)); - stages.add(stage); - } - - @Override - public Iterator iterator() { - return stages.iterator(); - } - - public void validate() { - setStages(getStages()); - for (Stage stage : stages) - stage.validate(); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.config; + +import java.util.*; + +import com.intel.cosbench.config.common.ConfigUtils; +import com.intel.cosbench.log.LogFactory; +import com.intel.cosbench.log.Logger; + +/** + * The model class mapping to "workflow" in configuration xml with following form: + * + * + * @author ywang19, qzheng7 + * + */ +public class Workflow implements Iterable { + + private List stages; + private String config = ""; + + public Workflow() { + /* empty */ + } + + public String getConfig() { + return config; + } + + public void setConfig(String config) { + this.config = config; + } + + public List getStages() { + return stages; + } + + public void setStages(List stages) { + if (stages == null || stages.isEmpty()) + throw new ConfigException("workflow must have stages"); + for(Stage stage: stages) { + stage.setConfig(ConfigUtils.inherit(stage.getConfig(), this.config)); + } + this.stages = stages; + } + + public void addStage(Stage stage) { + if (stage == null || stage.getName().isEmpty()) + throw new ConfigException("can't add one empty stage"); + if (stages == null) + stages = new ArrayList(); + stage.setConfig(ConfigUtils.inherit(stage.getConfig(), this.config)); + stages.add(stage); + } + + @Override + public Iterator iterator() { + if(stages == null){ + stages = new ArrayList(); + } + return stages.iterator(); + } + + public void validate() { + setStages(getStages()); + for (Stage stage : stages) + stage.validate(); + } + +} diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/Workload.java b/dev/cosbench-config/src/com/intel/cosbench/config/Workload.java index c04b5b12..90a0df57 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/Workload.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/Workload.java @@ -1,129 +1,136 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.config; - -import org.apache.commons.lang.StringUtils; - -import com.intel.cosbench.config.common.ConfigUtils; -import com.intel.cosbench.log.LogFactory; -import com.intel.cosbench.log.Logger; - -/** - * The model class mapping to "workload" in configuration xml with following form: - * - * - * @author ywang19, qzheng7 - * - */ -public class Workload { - - private static final Auth DEFAULT_AUTH = new Auth("none"); - private static final Storage DEFAULT_STORAGE = new Storage("none"); - - private String name; - private String description; - private String trigger=null; - private String config = ""; - private Auth auth = DEFAULT_AUTH; - private Storage storage = DEFAULT_STORAGE; - private Workflow workflow; - - public Workload() { - /* empty */ - } - - public String getName() { - return name; - } - - public void setName(String name) { - if (StringUtils.isEmpty(name)) - throw new ConfigException("workload name cannot be empty"); - this.name = name; - } - - public String getDescription() { - return description; - } - - public void setDescription(String description) { - /* description might be empty */ - this.description = description; - } - - public String getTrigger() { - return trigger; - } - - public void setTrigger(String trigger) { - this.trigger = trigger; - } - - public String getConfig() { - return config; - } - - public void setConfig(String config) { - /* configuration might be empty */ - this.config = config; - } - - public Auth getAuth() { - return auth; - } - - public void setAuth(Auth auth) { - if (auth == null) - throw new ConfigException("workload must have a default auth"); - this.auth = auth; - } - - public Storage getStorage() { - return storage; - } - - public void setStorage(Storage storage) { - if (storage == null) - throw new ConfigException("workload must have a default storage"); - this.storage = storage; - } - - public Workflow getWorkflow() { - return workflow; - } - - public void setWorkflow(Workflow workflow) { - if (workflow == null) - throw new ConfigException("workload must have its workflow"); - workflow.setConfig(ConfigUtils.inherit(workflow.getConfig(), this.config)); - this.workflow = workflow; - } - - public void validate() { - setName(getName()); - setWorkflow(getWorkflow()); - for (Stage stage : workflow) - if (stage.getAuth() == null) - stage.setAuth(auth); - for (Stage stage : workflow) - if (stage.getStorage() == null) - stage.setStorage(storage); - workflow.validate(); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.config; + +import java.io.File; + +import org.apache.commons.lang.StringUtils; + +import com.intel.cosbench.config.common.ConfigUtils; +import com.intel.cosbench.log.LogFactory; +import com.intel.cosbench.log.Logger; + +/** + * The model class mapping to "workload" in configuration xml with following form: + * + * + * @author ywang19, qzheng7 + * + */ +public class Workload { + + private static final Auth DEFAULT_AUTH = new Auth("none"); + private static final Storage DEFAULT_STORAGE = new Storage("none"); + + private String name; + private String description; + private String trigger=null; + private String config = ""; + private Auth auth = DEFAULT_AUTH; + private Storage storage = DEFAULT_STORAGE; + private Workflow workflow; + + public Workload() { + /* empty */ + } + + public String getName() { + return name; + } + + public void setName(String name) { + if (StringUtils.isEmpty(name)) + throw new ConfigException("workload name cannot be empty"); + this.name = name; + } + + public String getDescription() { + return description; + } + + public void setDescription(String description) { + /* description might be empty */ + this.description = description; + } + + public String getTrigger() { + return trigger; + } + + public void setTrigger(String trigger) { + this.trigger = trigger; + } + + public String getConfig() { + return config; + } + + public void setConfig(String config) { + /* configuration might be empty */ + this.config = config; + } + + public Auth getAuth() { + return auth; + } + + public void setAuth(Auth auth) { + if (auth == null) + throw new ConfigException("workload must have a default auth"); + this.auth = auth; + } + + public Storage getStorage() { + return storage; + } + + public void setStorage(Storage storage) { + if (storage == null) + throw new ConfigException("workload must have a default storage"); + this.storage = storage; + } + + public Workflow getWorkflow() { + return workflow; + } + + public void setWorkflow(Workflow workflow) { + if (workflow == null) + throw new ConfigException("workload must have its workflow"); + workflow.setConfig(ConfigUtils.inherit(workflow.getConfig(), this.config)); + this.workflow = workflow; + } + + public void validate() { + if (getName().indexOf(File.separator) >= 0) { + throw new ConfigException("Forbidden character (" + File.separator + ") in the name of the workload."); + } + + setName(getName()); + setWorkflow(getWorkflow()); + for (Stage stage : workflow) + if (stage.getAuth() == null) + stage.setAuth(auth); + for (Stage stage : workflow) + if (stage.getStorage() == null) + stage.setStorage(storage); + workflow.validate(); + } + +} diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/WorkloadResolver.java b/dev/cosbench-config/src/com/intel/cosbench/config/WorkloadResolver.java index d3a6eeb8..79f2167c 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/WorkloadResolver.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/WorkloadResolver.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/WorkloadWriter.java b/dev/cosbench-config/src/com/intel/cosbench/config/WorkloadWriter.java index d22f0bc9..086efc01 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/WorkloadWriter.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/WorkloadWriter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/XmlConfig.java b/dev/cosbench-config/src/com/intel/cosbench/config/XmlConfig.java index 4d02aabd..496df0f1 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/XmlConfig.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/XmlConfig.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config; @@ -22,7 +22,7 @@ /** * The wrapper class to handle xml based configuration. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigBase.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigBase.java index 8e4c365c..81d1b604 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigBase.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigBase.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; @@ -30,7 +30,7 @@ /** * Base class for castor configuration. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigResolver.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigResolver.java index 69b285db..5e452362 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigResolver.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigResolver.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; @@ -25,7 +25,7 @@ /** * Abstract class for Castor resolver, this class converts xml configuration into domain object. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigTools.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigTools.java index 54f7479b..75588a96 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigTools.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigTools.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigWriter.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigWriter.java index 2ab86d51..9600f6f8 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigWriter.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorConfigWriter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; @@ -25,7 +25,7 @@ /** * Class for castor configuration writer, this class writes domain object to xml configuration. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMappings.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMappings.java index abae046c..15cb50a0 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMappings.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMappings.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; @@ -22,7 +22,7 @@ /** * the collection of model mapping files - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMissionResolver.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMissionResolver.java index 030d25f5..83384a09 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMissionResolver.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMissionResolver.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; @@ -23,7 +23,7 @@ /** * The castor resolver for mission object, it resolves mission object from xml configuration. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMissionWriter.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMissionWriter.java index 022167b6..9e296fab 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMissionWriter.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorMissionWriter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; @@ -21,7 +21,7 @@ /** * The castor writer for mission object, this class writes mission to xml configuration. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorWorkloadResolver.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorWorkloadResolver.java index 205f3734..c4c39366 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorWorkloadResolver.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorWorkloadResolver.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; @@ -23,7 +23,7 @@ /** * The castor resolver for workload object, it resolves workload object from xml configuration. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorWorkloadWriter.java b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorWorkloadWriter.java index 569bdb5a..896ae2c3 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorWorkloadWriter.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/castor/CastorWorkloadWriter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.castor; @@ -21,7 +21,7 @@ /** * The castor writer for workload object, this class writes workload to xml configuration. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/common/COSBConfigApator.java b/dev/cosbench-config/src/com/intel/cosbench/config/common/COSBConfigApator.java index 0c31891d..a17cd44e 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/common/COSBConfigApator.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/common/COSBConfigApator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.common; @@ -23,7 +23,7 @@ /** * One COSBench implementation of Config interface, which encapsulates differences of different configuration format. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/common/ConfigUtils.java b/dev/cosbench-config/src/com/intel/cosbench/config/common/ConfigUtils.java index c31000ab..f866205a 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/common/ConfigUtils.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/common/ConfigUtils.java @@ -1,17 +1,35 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.config.common; import com.intel.cosbench.config.ConfigConstants; public class ConfigUtils { - public static String inherit(String child_config, String parent_config) { - if(parent_config.equals("")) { - return child_config; - } - if(child_config.equals("")) { - return parent_config; - } - child_config = parent_config + ConfigConstants.DELIMITER + child_config; - return child_config; - } + public static String inherit(String child_config, String parent_config) { + if(parent_config.equals("")) { + return child_config; + } + if(child_config.equals("")) { + return parent_config; + } + child_config = parent_config + ConfigConstants.DELIMITER + child_config; + return child_config; + } } diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/common/INIConfigParser.java b/dev/cosbench-config/src/com/intel/cosbench/config/common/INIConfigParser.java index f4cf7981..1a7e078e 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/common/INIConfigParser.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/common/INIConfigParser.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.common; @@ -27,7 +27,7 @@ /** * The parser for INI format configuration. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-config/src/com/intel/cosbench/config/common/KVConfigParser.java b/dev/cosbench-config/src/com/intel/cosbench/config/common/KVConfigParser.java index 23650a54..72c55566 100644 --- a/dev/cosbench-config/src/com/intel/cosbench/config/common/KVConfigParser.java +++ b/dev/cosbench-config/src/com/intel/cosbench/config/common/KVConfigParser.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.config.common; @@ -25,7 +25,7 @@ /** * The Parser for key-value configuration. - * + * * @author ywang19, qzheng7 * */ @@ -48,7 +48,7 @@ private static void addConfigEntry(String entry, Configuration config) { int pos = StringUtils.indexOf(entry, '='); if(pos < 0) logger.warn("cannot parse config entry {}", entry); - + String key = StringUtils.trim(StringUtils.left(entry, pos)); String value = StringUtils.trim(StringUtils.right(entry, entry.length() - pos -1)); logger.debug("key=" + key + ";value=" + value); diff --git a/dev/cosbench-controller-web/META-INF/MANIFEST.MF b/dev/cosbench-controller-web/META-INF/MANIFEST.MF index d56a5b35..5ea5134d 100644 --- a/dev/cosbench-controller-web/META-INF/MANIFEST.MF +++ b/dev/cosbench-controller-web/META-INF/MANIFEST.MF @@ -3,7 +3,7 @@ Web-ContextPath: controller Bundle-ManifestVersion: 2 Bundle-Name: COSBench Controller Web Bundle Bundle-SymbolicName: cosbench-controller-web -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.bench, diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/400.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/400.ftl index 94213dee..3cc9448d 100644 --- a/dev/cosbench-controller-web/WEB-INF/freemarker/400.ftl +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/400.ftl @@ -6,7 +6,7 @@ 400 Bad Request -<#include "header.ftl"> +<#include "head.ftl">

diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/404.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/404.ftl index 81805a8a..3020c294 100644 --- a/dev/cosbench-controller-web/WEB-INF/freemarker/404.ftl +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/404.ftl @@ -6,12 +6,14 @@ 404 Not Found -<#include "header.ftl"> +<#include "head.ftl">

404 Not Found

+<#if resource?? >

Note: The resource [ ${resource} ] you are looking for cannot be found!

+

go back to index

<#-- end of content -->

diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/500.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/500.ftl index 6647775e..8e732cf0 100644 --- a/dev/cosbench-controller-web/WEB-INF/freemarker/500.ftl +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/500.ftl @@ -6,7 +6,7 @@ Internal Error -<#include "header.ftl"> +<#include "head.ftl">

diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/finalchart.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/finalchart.ftl new file mode 100644 index 00000000..bf4d3d72 --- /dev/null +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/finalchart.ftl @@ -0,0 +1,129 @@ + \ No newline at end of file diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/forchart.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/forchart.ftl new file mode 100644 index 00000000..4c78fd1a --- /dev/null +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/forchart.ftl @@ -0,0 +1,82 @@ + + \ No newline at end of file diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/head.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/head.ftl new file mode 100644 index 00000000..c745d920 --- /dev/null +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/head.ftl @@ -0,0 +1,5 @@ + <#-- end of header --> diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/header.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/header.ftl index 2b9b4311..7ec86cf0 100644 --- a/dev/cosbench-controller-web/WEB-INF/freemarker/header.ftl +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/header.ftl @@ -1,6 +1,6 @@ <#-- end of header --> diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/metrics.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/metrics.ftl index 3c66caff..e7c0d1a4 100644 --- a/dev/cosbench-controller-web/WEB-INF/freemarker/metrics.ftl +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/metrics.ftl @@ -12,7 +12,11 @@ <#list allMetrics as mInfo> - ${mInfo.opName}<#if mInfo.opName != mInfo.sampleType>-${mInfo.sampleType} + + <#if mInfo.opId??>${mInfo.opId}: + ${mInfo.opName} + <#if mInfo.opName != mInfo.sampleType>-${mInfo.sampleType} + <#assign op = mInfo.sampleCount > <#if (op >= 1000) > diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/runningchart.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/runningchart.ftl new file mode 100644 index 00000000..07ec5561 --- /dev/null +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/runningchart.ftl @@ -0,0 +1,117 @@ + \ No newline at end of file diff --git a/dev/cosbench-controller-web/WEB-INF/freemarker/workload.ftl b/dev/cosbench-controller-web/WEB-INF/freemarker/workload.ftl index 006745a2..22b88ffd 100644 --- a/dev/cosbench-controller-web/WEB-INF/freemarker/workload.ftl +++ b/dev/cosbench-controller-web/WEB-INF/freemarker/workload.ftl @@ -10,6 +10,7 @@ + Workload Details @@ -203,6 +204,22 @@ + + +<#if isRunning> +

Performance Graph

+
+
+
+
+
+
+ <#include "forchart.ftl"> + <#include "runningchart.ftl"> + +
+ +

There are ${info.stageCount} stages in this workload.

<#if isStopped> <#if showErrorStatistics> @@ -211,7 +228,7 @@ Driver Url Error Code - Occurence Number + Occurrence Number <#list info.errorStatistics?keys as url> <#list info.errorStatistics[url].errorCodeAndNum?keys as code> @@ -231,7 +248,28 @@ <#else>

show error statistics details

+ +

Performance Graph + +

+ + +

Actions

<#if !isStopped > diff --git a/dev/cosbench-controller-web/WEB-INF/spring/controller-web-context.xml b/dev/cosbench-controller-web/WEB-INF/spring/controller-web-context.xml index 78cceeb8..cd324ecb 100644 --- a/dev/cosbench-controller-web/WEB-INF/spring/controller-web-context.xml +++ b/dev/cosbench-controller-web/WEB-INF/spring/controller-web-context.xml @@ -91,6 +91,11 @@ + + + + diff --git a/dev/cosbench-controller-web/WEB-INF/web.xml b/dev/cosbench-controller-web/WEB-INF/web.xml index cc58f131..28a147b3 100644 --- a/dev/cosbench-controller-web/WEB-INF/web.xml +++ b/dev/cosbench-controller-web/WEB-INF/web.xml @@ -35,6 +35,10 @@ main *.csv + + main + /prometheus + client @@ -71,6 +75,10 @@ resource *.css + + resource + *.js + resource *.png @@ -101,6 +109,10 @@ css text/css + + js + text/javascript + png image/png diff --git a/dev/cosbench-controller-web/resources/build/dist/chart/bar.js b/dev/cosbench-controller-web/resources/build/dist/chart/bar.js new file mode 100644 index 00000000..5925f3fc --- /dev/null +++ b/dev/cosbench-controller-web/resources/build/dist/chart/bar.js @@ -0,0 +1 @@ +define("echarts/chart/bar",["require","../component/base","./base","zrender/shape/Rectangle","../component/axis","../component/grid","../component/dataZoom","../config","../util/ecData","zrender/tool/util","zrender/tool/color","../chart"],function(e){function t(e,t,a,o,r){i.call(this,e,t,a,o,r),n.call(this),this.refresh(o)}var i=e("../component/base"),n=e("./base"),a=e("zrender/shape/Rectangle");e("../component/axis"),e("../component/grid"),e("../component/dataZoom");var o=e("../config"),r=e("../util/ecData"),s=e("zrender/tool/util"),l=e("zrender/tool/color");return t.prototype={type:o.CHART_TYPE_BAR,_buildShape:function(){this._bulidPosition()},_buildNormal:function(e,t,i,n,o){for(var r,s,l,h,d,m,c,p,u,V,U,y,g=this.series,f=i[0][0],b=g[f],_=b.xAxisIndex,x=b.yAxisIndex,k="horizontal"==o?this.component.xAxis.getAxis(_):this.component.yAxis.getAxis(x),L=this._mapSize(k,i),v=L.gap,W=L.barGap,w=L.barWidthMap,X=L.barMaxWidthMap,I=L.barWidth,K=L.barMinHeightMap,S=L.interval,T=0,C=t;C>T&&null!=k.getNameByIndex(T);T++){"horizontal"==o?h=k.getCoordByIndex(T)-v/2:d=k.getCoordByIndex(T)+v/2;for(var E=0,A=i.length;A>E;E++){x=g[i[E][0]].yAxisIndex||0,_=g[i[E][0]].xAxisIndex||0,r="horizontal"==o?this.component.yAxis.getAxis(x):this.component.xAxis.getAxis(_),c=m=u=p=r.getCoord(0);for(var z=0,M=i[E].length;M>z;z++)if(f=i[E][z],b=g[f],U=b.data[T],y=null!=U?null!=U.value?U.value:U:"-",n[f]=n[f]||{min:Number.POSITIVE_INFINITY,max:Number.NEGATIVE_INFINITY,sum:0,counter:0,average:0},"-"!==y){y>0?(s=z>0?r.getCoordSize(y):"horizontal"==o?c-r.getCoord(y):r.getCoord(y)-c,1===M&&K[f]>s&&(s=K[f]),"horizontal"==o?(m-=s,d=m):(h=m,m+=s)):0>y?(s=z>0?r.getCoordSize(y):"horizontal"==o?r.getCoord(y)-u:u-r.getCoord(y),1===M&&K[f]>s&&(s=K[f]),"horizontal"==o?(d=p,p+=s):(p-=s,h=p)):(s=0,"horizontal"==o?(m-=s,d=m):(h=m,m+=s));var l=Math.min(X[f]||Number.MAX_VALUE,w[f]||I);n[f][T]="horizontal"==o?h+l/2:d-l/2,n[f].min>y&&(n[f].min=y,"horizontal"==o?(n[f].minY=d,n[f].minX=n[f][T]):(n[f].minX=h+s,n[f].minY=n[f][T])),n[f].maxz;z++)f=i[E][z],b=g[f],U=b.data[T],y=null!=U?null!=U.value?U.value:U:"-","-"==y&&this.deepQuery([U,b,this.option],"calculable")&&("horizontal"==o?(m-=this.ecTheme.island.r,d=m):(h=m,m+=this.ecTheme.island.r),l=Math.min(X[f]||Number.MAX_VALUE,w[f]||I),V=this._getBarItem(f,T,k.getNameByIndex(T),h+.5,d+.5-("horizontal"==o?0:l),("horizontal"==o?l:this.ecTheme.island.r)-1,("horizontal"==o?this.ecTheme.island.r:l)-1,"horizontal"==o?"vertical":"horizontal"),V.hoverable=!1,V.draggable=!1,V.style.lineWidth=1,V.style.brushType="stroke",V.style.strokeColor=b.calculableHolderColor||this.ecTheme.calculableHolderColor,this.shapeList.push(new a(V)));"horizontal"==o?h+=l+W:d-=l+W}}this._calculMarkMapXY(n,i,"horizontal"==o?"y":"x")},_buildHorizontal:function(e,t,i,n){return this._buildNormal(e,t,i,n,"horizontal")},_buildVertical:function(e,t,i,n){return this._buildNormal(e,t,i,n,"vertical")},_buildOther:function(e,t,i,n){for(var o=this.series,r=0,s=i.length;s>r;r++)for(var l=0,h=i[r].length;h>l;l++){var d=i[r][l],m=o[d],c=m.xAxisIndex||0,p=this.component.xAxis.getAxis(c),u=p.getCoord(0),V=m.yAxisIndex||0,U=this.component.yAxis.getAxis(V),y=U.getCoord(0);n[d]=n[d]||{min0:Number.POSITIVE_INFINITY,min1:Number.POSITIVE_INFINITY,max0:Number.NEGATIVE_INFINITY,max1:Number.NEGATIVE_INFINITY,sum0:0,sum1:0,counter0:0,counter1:0,average0:0,average1:0};for(var g=0,f=m.data.length;f>g;g++){var b=m.data[g],_=null!=b?null!=b.value?b.value:b:"-";if(_ instanceof Array){var x,k,L=p.getCoord(_[0]),v=U.getCoord(_[1]),W=[b,m],w=this.deepQuery(W,"barWidth")||10,X=this.deepQuery(W,"barHeight");null!=X?(x="horizontal",_[0]>0?(w=L-u,L-=w):w=_[0]<0?u-L:0,k=this._getBarItem(d,g,_[0],L,v-X/2,w,X,x)):(x="vertical",_[1]>0?X=y-v:_[1]<0?(X=v-y,v-=X):X=0,k=this._getBarItem(d,g,_[0],L-w/2,v,w,X,x)),this.shapeList.push(new a(k)),L=p.getCoord(_[0]),v=U.getCoord(_[1]),n[d].min0>_[0]&&(n[d].min0=_[0],n[d].minY0=v,n[d].minX0=L),n[d].max0<_[0]&&(n[d].max0=_[0],n[d].maxY0=v,n[d].maxX0=L),n[d].sum0+=_[0],n[d].counter0++,n[d].min1>_[1]&&(n[d].min1=_[1],n[d].minY1=v,n[d].minX1=L),n[d].max1<_[1]&&(n[d].max1=_[1],n[d].maxY1=v,n[d].maxX1=L),n[d].sum1+=_[1],n[d].counter1++}}}this._calculMarkMapXY(n,i,"xy")},_mapSize:function(e,t,i){var n,a,o=this._findSpecialBarSzie(t,i),r=o.barWidthMap,s=o.barMaxWidthMap,l=o.barMinHeightMap,h=o.sBarWidthCounter,d=o.sBarWidthTotal,m=o.barGap,c=o.barCategoryGap,p=1;if(t.length!=h){if(i)n=e.getGap(),m=0,a=Math.floor(n/t.length),0>=a&&(p=Math.floor(t.length/n),a=1);else if(n="string"==typeof c&&c.match(/%$/)?Math.floor(e.getGap()*(100-parseFloat(c))/100):e.getGap()-c,"string"==typeof m&&m.match(/%$/)?(m=parseFloat(m)/100,a=Math.floor((n-d)/((t.length-1)*m+t.length-h)),m=Math.floor(a*m)):(m=parseFloat(m),a=Math.floor((n-d-m*(t.length-1))/(t.length-h))),0>=a)return this._mapSize(e,t,!0)}else if(n=h>1?"string"==typeof c&&c.match(/%$/)?Math.floor(e.getGap()*(100-parseFloat(c))/100):e.getGap()-c:d,a=0,m=h>1?Math.floor((n-d)/(h-1)):0,0>m)return this._mapSize(e,t,!0);return this._recheckBarMaxWidth(t,r,s,l,n,a,m,p)},_findSpecialBarSzie:function(e,t){for(var i,n,a,o,r=this.series,s={},l={},h={},d=0,m=0,c=0,p=e.length;p>c;c++)for(var u={barWidth:!1,barMaxWidth:!1},V=0,U=e[c].length;U>V;V++){var y=e[c][V],g=r[y];if(!t){if(u.barWidth)s[y]=i;else if(i=this.query(g,"barWidth"),null!=i){s[y]=i,m+=i,d++,u.barWidth=!0;for(var f=0,b=V;b>f;f++){var _=e[c][f];s[_]=i}}if(u.barMaxWidth)l[y]=n;else if(n=this.query(g,"barMaxWidth"),null!=n){l[y]=n,u.barMaxWidth=!0;for(var f=0,b=V;b>f;f++){var _=e[c][f];l[_]=n}}}h[y]=this.query(g,"barMinHeight"),a=null!=a?a:this.query(g,"barGap"),o=null!=o?o:this.query(g,"barCategoryGap")}return{barWidthMap:s,barMaxWidthMap:l,barMinHeightMap:h,sBarWidth:i,sBarMaxWidth:n,sBarWidthCounter:d,sBarWidthTotal:m,barGap:a,barCategoryGap:o}},_recheckBarMaxWidth:function(e,t,i,n,a,o,r,s){for(var l=0,h=e.length;h>l;l++){var d=e[l][0];i[d]&&i[d]0&&d.style.height>f&&d.style.width>f?(d.style.y+=f/2,d.style.height-=f,d.style.x+=f/2,d.style.width-=f):d.style.brushType="fill",d.highlightStyle.textColor=d.highlightStyle.color,d=this.addLabel(d,c,p,i,h),"insideLeft"===d.style.textPosition||"insideRight"===d.style.textPosition||"insideTop"===d.style.textPosition||"insideBottom"===d.style.textPosition){var _=5;switch(d.style.textPosition){case"insideLeft":d.style.textX=d.style.x+_,d.style.textY=d.style.y+d.style.height/2,d.style.textAlign="left",d.style.textBaseline="middle";break;case"insideRight":d.style.textX=d.style.x+d.style.width-_,d.style.textY=d.style.y+d.style.height/2,d.style.textAlign="right",d.style.textBaseline="middle";break;case"insideTop":d.style.textX=d.style.x+d.style.width/2,d.style.textY=d.style.y+_/2,d.style.textAlign="center",d.style.textBaseline="top";break;case"insideBottom":d.style.textX=d.style.x+d.style.width/2,d.style.textY=d.style.y+d.style.height-_/2,d.style.textAlign="center",d.style.textBaseline="bottom"}d.style.textPosition="specific",d.style.textColor=d.style.textColor||"#fff"}return this.deepQuery([p,c,this.option],"calculable")&&(this.setCalculable(d),d.draggable=!0),r.pack(d,m[e],e,m[e].data[t],t,i),d},getMarkCoord:function(e,t){var i,n,a=this.series[e],o=this.xMarkMap[e],r=this.component.xAxis.getAxis(a.xAxisIndex),s=this.component.yAxis.getAxis(a.yAxisIndex);if(!t.type||"max"!==t.type&&"min"!==t.type&&"average"!==t.type)if(o.isHorizontal){i="string"==typeof t.xAxis&&r.getIndexByName?r.getIndexByName(t.xAxis):t.xAxis||0;var l=o[i];l=null!=l?l:"string"!=typeof t.xAxis&&r.getCoordByIndex?r.getCoordByIndex(t.xAxis||0):r.getCoord(t.xAxis||0),n=[l,s.getCoord(t.yAxis||0)]}else{i="string"==typeof t.yAxis&&s.getIndexByName?s.getIndexByName(t.yAxis):t.yAxis||0;var h=o[i];h=null!=h?h:"string"!=typeof t.yAxis&&s.getCoordByIndex?s.getCoordByIndex(t.yAxis||0):s.getCoord(t.yAxis||0),n=[r.getCoord(t.xAxis||0),h]}else{var d=null!=t.valueIndex?t.valueIndex:null!=o.maxX0?"1":"";n=[o[t.type+"X"+d],o[t.type+"Y"+d],o[t.type+"Line"+d],o[t.type+d]]}return n},refresh:function(e){e&&(this.option=e,this.series=e.series),this.backupShapeList(),this._buildShape()},addDataAnimation:function(e){for(var t=this.series,i={},n=0,a=e.length;a>n;n++)i[e[n][0]]=e[n];for(var o,s,l,h,d,m,c,n=this.shapeList.length-1;n>=0;n--)if(m=r.get(this.shapeList[n],"seriesIndex"),i[m]&&!i[m][3]&&"rectangle"===this.shapeList[n].type){if(c=r.get(this.shapeList[n],"dataIndex"),d=t[m],i[m][2]&&c===d.data.length-1){this.zr.delShape(this.shapeList[n].id);continue}if(!i[m][2]&&0===c){this.zr.delShape(this.shapeList[n].id);continue}"horizontal"===this.shapeList[n]._orient?(h=this.component.yAxis.getAxis(d.yAxisIndex||0).getGap(),l=i[m][2]?-h:h,o=0):(s=this.component.xAxis.getAxis(d.xAxisIndex||0).getGap(),o=i[m][2]?s:-s,l=0),this.shapeList[n].position=[0,0],this.zr.animate(this.shapeList[n].id,"").when(500,{position:[o,l]}).start()}}},s.inherits(t,n),s.inherits(t,i),e("../chart").define("bar",t),t}); \ No newline at end of file diff --git a/dev/cosbench-controller-web/resources/build/dist/chart/line.js b/dev/cosbench-controller-web/resources/build/dist/chart/line.js new file mode 100644 index 00000000..da2a590a --- /dev/null +++ b/dev/cosbench-controller-web/resources/build/dist/chart/line.js @@ -0,0 +1 @@ +define("echarts/chart/line",["require","../component/base","./base","zrender/shape/BrokenLine","../util/shape/Icon","../util/shape/HalfSmoothPolygon","../component/axis","../component/grid","../component/dataZoom","../config","../util/ecData","zrender/tool/util","zrender/tool/color","../chart"],function(e){function t(e,t,i,o,r){n.call(this,e,t,i,o,r),a.call(this),this.refresh(o)}function i(e,t,i){var n=t.x,a=t.y,o=t.width,s=t.height,l=s/2;t.symbol.match("empty")&&(e.fillStyle="#fff"),t.brushType="both";var h=t.symbol.replace("empty","").toLowerCase();h.match("star")?(l=h.replace("star","")-0||5,a-=1,h="star"):("rectangle"===h||"arrow"===h)&&(n+=(o-s)/2,o=s);var d="";if(h.match("image")&&(d=h.replace(new RegExp("^image:\\/\\/"),""),h="image",n+=Math.round((o-s)/2)-1,o=s+=2),h=r.prototype.iconLibrary[h]){var m=t.x,c=t.y;e.moveTo(m,c+l),e.lineTo(m+5,c+l),e.moveTo(m+t.width-5,c+l),e.lineTo(m+t.width,c+l);var p=this;h(e,{x:n+4,y:a+4,width:o-8,height:s-8,n:l,image:d},function(){p.modSelf(),i()})}else e.moveTo(n,a+l),e.lineTo(n+o,a+l)}var n=e("../component/base"),a=e("./base"),o=e("zrender/shape/BrokenLine"),r=e("../util/shape/Icon"),s=e("../util/shape/HalfSmoothPolygon");e("../component/axis"),e("../component/grid"),e("../component/dataZoom");var l=e("../config"),h=e("../util/ecData"),d=e("zrender/tool/util"),m=e("zrender/tool/color");return t.prototype={type:l.CHART_TYPE_LINE,_buildShape:function(){this.finalPLMap={},this._bulidPosition()},_buildHorizontal:function(e,t,i,n){for(var a,o,r,s,l,h,d,m,c,p,u=this.series,V=i[0][0],U=u[V],y=U.xAxisIndex,g=this.component.xAxis.getAxis(y),f={},b=0,_=t;_>b&&null!=g.getNameByIndex(b);b++){r=g.getCoordByIndex(b);for(var x=0,k=i.length;k>x;x++){a=u[i[x][0]].yAxisIndex||0,o=this.component.yAxis.getAxis(a),h=l=m=d=o.getCoord(0);for(var L=0,v=i[x].length;v>L;L++)V=i[x][L],U=u[V],c=U.data[b],p=null!=c?null!=c.value?c.value:c:"-",f[V]=f[V]||[],n[V]=n[V]||{min:Number.POSITIVE_INFINITY,max:Number.NEGATIVE_INFINITY,sum:0,counter:0,average:0},"-"!==p?(p>=0?(l-=L>0?o.getCoordSize(p):h-o.getCoord(p),s=l):0>p&&(d+=L>0?o.getCoordSize(p):o.getCoord(p)-m,s=d),f[V].push([r,s,b,g.getNameByIndex(b),r,h]),n[V].min>p&&(n[V].min=p,n[V].minY=s,n[V].minX=r),n[V].max0&&(this.finalPLMap[V]=this.finalPLMap[V]||[],this.finalPLMap[V].push(f[V]),f[V]=[])}l=this.component.grid.getY();for(var W,x=0,k=i.length;k>x;x++)for(var L=0,v=i[x].length;v>L;L++)V=i[x][L],U=u[V],c=U.data[b],p=null!=c?null!=c.value?c.value:c:"-","-"==p&&this.deepQuery([c,U,this.option],"calculable")&&(W=this.deepQuery([c,U],"symbolSize"),l+=2*W+5,s=l,this.shapeList.push(this._getCalculableItem(V,b,g.getNameByIndex(b),r,s,"horizontal")))}for(var w in f)f[w].length>0&&(this.finalPLMap[w]=this.finalPLMap[w]||[],this.finalPLMap[w].push(f[w]),f[w]=[]);this._calculMarkMapXY(n,i,"y"),this._buildBorkenLine(e,this.finalPLMap,g,"horizontal")},_buildVertical:function(e,t,i,n){for(var a,o,r,s,l,h,d,m,c,p,u=this.series,V=i[0][0],U=u[V],y=U.yAxisIndex,g=this.component.yAxis.getAxis(y),f={},b=0,_=t;_>b&&null!=g.getNameByIndex(b);b++){s=g.getCoordByIndex(b);for(var x=0,k=i.length;k>x;x++){a=u[i[x][0]].xAxisIndex||0,o=this.component.xAxis.getAxis(a),h=l=m=d=o.getCoord(0);for(var L=0,v=i[x].length;v>L;L++)V=i[x][L],U=u[V],c=U.data[b],p=null!=c?null!=c.value?c.value:c:"-",f[V]=f[V]||[],n[V]=n[V]||{min:Number.POSITIVE_INFINITY,max:Number.NEGATIVE_INFINITY,sum:0,counter:0,average:0},"-"!==p?(p>=0?(l+=L>0?o.getCoordSize(p):o.getCoord(p)-h,r=l):0>p&&(d-=L>0?o.getCoordSize(p):m-o.getCoord(p),r=d),f[V].push([r,s,b,g.getNameByIndex(b),h,s]),n[V].min>p&&(n[V].min=p,n[V].minX=r,n[V].minY=s),n[V].max0&&(this.finalPLMap[V]=this.finalPLMap[V]||[],this.finalPLMap[V].push(f[V]),f[V]=[])}l=this.component.grid.getXend();for(var W,x=0,k=i.length;k>x;x++)for(var L=0,v=i[x].length;v>L;L++)V=i[x][L],U=u[V],c=U.data[b],p=null!=c?null!=c.value?c.value:c:"-","-"==p&&this.deepQuery([c,U,this.option],"calculable")&&(W=this.deepQuery([c,U],"symbolSize"),l-=2*W+5,r=l,this.shapeList.push(this._getCalculableItem(V,b,g.getNameByIndex(b),r,s,"vertical")))}for(var w in f)f[w].length>0&&(this.finalPLMap[w]=this.finalPLMap[w]||[],this.finalPLMap[w].push(f[w]),f[w]=[]);this._calculMarkMapXY(n,i,"x"),this._buildBorkenLine(e,this.finalPLMap,g,"vertical")},_buildOther:function(e,t,i,n){for(var a,o,r=this.series,s={},l=0,h=i.length;h>l;l++)for(var d=0,m=i[l].length;m>d;d++){var c=i[l][d],p=r[c],u=p.xAxisIndex||0;a=this.component.xAxis.getAxis(u);var V=p.yAxisIndex||0;o=this.component.yAxis.getAxis(V);var U=o.getCoord(0);s[c]=s[c]||[],n[c]=n[c]||{min0:Number.POSITIVE_INFINITY,min1:Number.POSITIVE_INFINITY,max0:Number.NEGATIVE_INFINITY,max1:Number.NEGATIVE_INFINITY,sum0:0,sum1:0,counter0:0,counter1:0,average0:0,average1:0};for(var y=0,g=p.data.length;g>y;y++){var f=p.data[y],b=null!=f?null!=f.value?f.value:f:"-";if(b instanceof Array){var _=a.getCoord(b[0]),x=o.getCoord(b[1]);s[c].push([_,x,y,b[0],_,U]),n[c].min0>b[0]&&(n[c].min0=b[0],n[c].minY0=x,n[c].minX0=_),n[c].max0b[1]&&(n[c].min1=b[1],n[c].minY1=x,n[c].minX1=_),n[c].max10&&(this.finalPLMap[k]=this.finalPLMap[k]||[],this.finalPLMap[k].push(s[k]),s[k]=[]);this._calculMarkMapXY(n,i,"xy"),this._buildBorkenLine(e,this.finalPLMap,a,"other")},_buildBorkenLine:function(e,t,i,n){for(var a,r="other"==n?"horizontal":n,l=this.series,c=e.length-1;c>=0;c--){var p=e[c],u=l[p],V=t[p];if(u.type===this.type&&null!=V)for(var U=this._getBbox(p,r),y=this._sIndex2ColorMap[p],g=this.query(u,"itemStyle.normal.lineStyle.width"),f=this.query(u,"itemStyle.normal.lineStyle.type"),b=this.query(u,"itemStyle.normal.lineStyle.color"),_=this.getItemStyleColor(this.query(u,"itemStyle.normal.color"),p,-1),x=null!=this.query(u,"itemStyle.normal.areaStyle"),k=this.query(u,"itemStyle.normal.areaStyle.color"),L=0,v=V.length;v>L;L++){var W=V[L],w="other"!=n&&this._isLarge(r,W);if(w)W=this._getLargePointList(r,W);else for(var X=0,I=W.length;I>X;X++)a=u.data[W[X][2]],(this.deepQuery([a,u,this.option],"calculable")||this.deepQuery([a,u],"showAllSymbol")||"categoryAxis"===i.type&&i.isMainAxis(W[X][2])&&"none"!=this.deepQuery([a,u],"symbol"))&&this.shapeList.push(this._getSymbol(p,W[X][2],W[X][3],W[X][0],W[X][1],r));var K=new o({zlevel:this._zlevelBase,style:{miterLimit:g,pointList:W,strokeColor:b||_||y,lineWidth:g,lineType:f,smooth:this._getSmooth(u.smooth),smoothConstraint:U,shadowColor:this.query(u,"itemStyle.normal.lineStyle.shadowColor"),shadowBlur:this.query(u,"itemStyle.normal.lineStyle.shadowBlur"),shadowOffsetX:this.query(u,"itemStyle.normal.lineStyle.shadowOffsetX"),shadowOffsetY:this.query(u,"itemStyle.normal.lineStyle.shadowOffsetY")},hoverable:!1,_main:!0,_seriesIndex:p,_orient:r});if(h.pack(K,l[p],p,0,L,l[p].name),this.shapeList.push(K),x){var S=new s({zlevel:this._zlevelBase,style:{miterLimit:g,pointList:d.clone(W).concat([[W[W.length-1][4],W[W.length-1][5]],[W[0][4],W[0][5]]]),brushType:"fill",smooth:this._getSmooth(u.smooth),smoothConstraint:U,color:k?k:m.alpha(y,.5)},highlightStyle:{brushType:"fill"},hoverable:!1,_main:!0,_seriesIndex:p,_orient:r});h.pack(S,l[p],p,0,L,l[p].name),this.shapeList.push(S)}}}},_getBbox:function(e,t){var i=this.component.grid.getBbox(),n=this.xMarkMap[e];return null!=n.minX0?[[Math.min(n.minX0,n.maxX0,n.minX1,n.maxX1),Math.min(n.minY0,n.maxY0,n.minY1,n.maxY1)],[Math.max(n.minX0,n.maxX0,n.minX1,n.maxX1),Math.max(n.minY0,n.maxY0,n.minY1,n.maxY1)]]:("horizontal"===t?(i[0][1]=Math.min(n.minY,n.maxY),i[1][1]=Math.max(n.minY,n.maxY)):(i[0][0]=Math.min(n.minX,n.maxX),i[1][0]=Math.max(n.minX,n.maxX)),i)},_isLarge:function(e,t){return t.length<2?!1:"horizontal"===e?Math.abs(t[0][0]-t[1][0])<.5:Math.abs(t[0][1]-t[1][1])<.5},_getLargePointList:function(e,t){var i;i="horizontal"===e?this.component.grid.getWidth():this.component.grid.getHeight();for(var n=t.length,a=[],o=0;i>o;o++)a[o]=t[Math.floor(n/i*o)];return a},_getSmooth:function(e){return e?.3:0},_getCalculableItem:function(e,t,i,n,a,o){var r=this.series,s=r[e].calculableHolderColor||this.ecTheme.calculableHolderColor,l=this._getSymbol(e,t,i,n,a,o);return l.style.color=s,l.style.strokeColor=s,l.rotation=[0,0],l.hoverable=!1,l.draggable=!1,l.style.text=void 0,l},_getSymbol:function(e,t,i,n,a,o){var r=this.series,s=r[e],l=s.data[t],h=this.getSymbolShape(s,e,l,t,i,n,a,this._sIndex2ShapeMap[e],this._sIndex2ColorMap[e],"#fff","vertical"===o?"horizontal":"vertical");return h.zlevel=this._zlevelBase+1,this.deepQuery([l,s,this.option],"calculable")&&(this.setCalculable(h),h.draggable=!0),h},getMarkCoord:function(e,t){var i=this.series[e],n=this.xMarkMap[e],a=this.component.xAxis.getAxis(i.xAxisIndex),o=this.component.yAxis.getAxis(i.yAxisIndex);if(t.type&&("max"===t.type||"min"===t.type||"average"===t.type)){var r=null!=t.valueIndex?t.valueIndex:null!=n.maxX0?"1":"";return[n[t.type+"X"+r],n[t.type+"Y"+r],n[t.type+"Line"+r],n[t.type+r]]}return["string"!=typeof t.xAxis&&a.getCoordByIndex?a.getCoordByIndex(t.xAxis||0):a.getCoord(t.xAxis||0),"string"!=typeof t.yAxis&&o.getCoordByIndex?o.getCoordByIndex(t.yAxis||0):o.getCoord(t.yAxis||0)]},refresh:function(e){e&&(this.option=e,this.series=e.series),this.backupShapeList(),this._buildShape()},ontooltipHover:function(e,t){for(var i,n,a=e.seriesIndex,o=e.dataIndex,r=a.length;r--;)if(i=this.finalPLMap[a[r]])for(var s=0,l=i.length;l>s;s++){n=i[s];for(var h=0,d=n.length;d>h;h++)o===n[h][2]&&t.push(this._getSymbol(a[r],n[h][2],n[h][3],n[h][0],n[h][1],"horizontal"))}},addDataAnimation:function(e){for(var t=this.series,i={},n=0,a=e.length;a>n;n++)i[e[n][0]]=e[n];for(var o,r,s,l,h,d,m,n=this.shapeList.length-1;n>=0;n--)if(h=this.shapeList[n]._seriesIndex,i[h]&&!i[h][3]){if(this.shapeList[n]._main&&this.shapeList[n].style.pointList.length>1){if(d=this.shapeList[n].style.pointList,r=Math.abs(d[0][0]-d[1][0]),l=Math.abs(d[0][1]-d[1][1]),m="horizontal"===this.shapeList[n]._orient,i[h][2]){if("half-smooth-polygon"===this.shapeList[n].type){var c=d.length;this.shapeList[n].style.pointList[c-3]=d[c-2],this.shapeList[n].style.pointList[c-3][m?0:1]=d[c-4][m?0:1],this.shapeList[n].style.pointList[c-2]=d[c-1]}this.shapeList[n].style.pointList.pop(),m?(o=r,s=0):(o=0,s=-l)}else{if(this.shapeList[n].style.pointList.shift(),"half-smooth-polygon"===this.shapeList[n].type){var p=this.shapeList[n].style.pointList.pop();m?p[0]=d[0][0]:p[1]=d[0][1],this.shapeList[n].style.pointList.push(p)}m?(o=-r,s=0):(o=0,s=l)}this.zr.modShape(this.shapeList[n].id,{style:{pointList:this.shapeList[n].style.pointList}},!0)}else{if(i[h][2]&&this.shapeList[n]._dataIndex===t[h].data.length-1){this.zr.delShape(this.shapeList[n].id);continue}if(!i[h][2]&&0===this.shapeList[n]._dataIndex){this.zr.delShape(this.shapeList[n].id);continue}}this.shapeList[n].position=[0,0],this.zr.animate(this.shapeList[n].id,"").when(500,{position:[o,s]}).start()}}},r.prototype.iconLibrary.legendLineIcon=i,d.inherits(t,a),d.inherits(t,n),e("../chart").define("line",t),t}),define("echarts/util/shape/HalfSmoothPolygon",["require","zrender/shape/Base","zrender/shape/util/smoothBezier","zrender/tool/util","zrender/shape/Polygon"],function(e){function t(e){i.call(this,e)}var i=e("zrender/shape/Base"),n=e("zrender/shape/util/smoothBezier"),a=e("zrender/tool/util");return t.prototype={type:"half-smooth-polygon",buildPath:function(t,i){var a=i.pointList;if(!(a.length<2))if(i.smooth){var o=n(a.slice(0,-2),i.smooth,!1,i.smoothConstraint);t.moveTo(a[0][0],a[0][1]);for(var r,s,l,h=a.length,d=0;h-3>d;d++)r=o[2*d],s=o[2*d+1],l=a[d+1],t.bezierCurveTo(r[0],r[1],s[0],s[1],l[0],l[1]);t.lineTo(a[h-2][0],a[h-2][1]),t.lineTo(a[h-1][0],a[h-1][1]),t.lineTo(a[0][0],a[0][1])}else e("zrender/shape/Polygon").prototype.buildPath(t,i)}},a.inherits(t,i),t}); \ No newline at end of file diff --git a/dev/cosbench-controller-web/resources/build/dist/echarts.js b/dev/cosbench-controller-web/resources/build/dist/echarts.js new file mode 100644 index 00000000..7e9e1a51 --- /dev/null +++ b/dev/cosbench-controller-web/resources/build/dist/echarts.js @@ -0,0 +1,11 @@ +var define,require,esl;!function(e){function t(e,t){function n(e){0===e.indexOf(".")&&a.push(e)}var a=[];if("string"==typeof e?n(e):T(e,function(e){n(e)}),a.length>0)throw new Error("[REQUIRE_FATAL]Relative ID is not allowed in global require: "+a.join(", "));var o=G.waitSeconds;return o&&e instanceof Array&&(E&&clearTimeout(E),E=setTimeout(i,1e3*o)),P(e,t)}function i(){function e(r,s){if(!o[r]&&!d(r,O)){o[r]=1,d(r,F)||n[r]||(n[r]=1,t.push(r));var l=A[r];l?s&&(n[r]||(n[r]=1,t.push(r)),T(l.depMs,function(t){e(t.absId,t.hard)})):a[r]||(a[r]=1,i.push(r))}}var t=[],i=[],n={},a={},o={};for(var r in z)e(r,1);if(t.length||i.length)throw new Error("[MODULE_TIMEOUT]Hang( "+(t.join(", ")||"none")+" ) Miss( "+(i.join(", ")||"none")+" )")}function n(e,t,i){if(null==i&&(null==t?(i=e,e=null):(i=t,t=null,e instanceof Array&&(t=e,e=null))),null!=i){var n=window.opera;if(!e&&document.attachEvent&&(!n||"[object Opera]"!==n.toString())){var a=I();e=a&&a.getAttribute("data-require-id")}e?(o(e,t,i),D&&clearTimeout(D)):B[0]={deps:t,factory:i}}}function a(){var e=G.config[this.id];return e&&"object"==typeof e?e:{}}function o(e,t,i){A[e]||(A[e]={id:e,depsDec:t,deps:t||["require","exports","module"],factoryDeps:[],factory:i,exports:{},config:a,state:M,require:L(e),depMs:[],depMkv:{},depRs:[],depPMs:[]})}function r(e){var t=A[e];if(t&&!d(e,J)){var i=t.deps,n=t.factory,a=0;"function"==typeof n&&(a=Math.min(n.length,i.length),!t.depsDec&&n.toString().replace(/(\/\*([\s\S]*?)\*\/|([^:]|^)\/\/(.*)$)/gm,"").replace(/require\(\s*(['"'])([^'"]+)\1\s*\)/g,function(e,t,n){i.push(n)}));var o=[];T(i,function(i,n){var r,s,l=w(i),h=v(l.mod,e);h&&!H[h]?(l.res&&(s={id:i,mod:h,res:l.res},z[h]=1,t.depPMs.push(h),t.depRs.push(s)),r=t.depMkv[h],r||(r={id:l.mod,absId:h,hard:a>n},t.depMs.push(r),t.depMkv[h]=r,o.push(h))):r={absId:h},a>n&&t.factoryDeps.push(s||r)}),t.state=J,h(e),y(o)}}function s(){for(var e in z)l(e),m(e)}function l(e){function t(e){if(!d(e,J))return!1;if(d(e,F)||i[e])return!0;i[e]=1;var n=A[e],a=!0;return T(n.depMs,function(e){return a=t(e.absId)}),a&&T(n.depRs,function(e){return a=!(!e.absId||!d(e.absId,O))}),a&&(n.state=F),a}var i={};t(e)}function h(t){function i(){if(!n&&a.state===F){n=1;var i=1,o=[];if(T(a.factoryDeps,function(e){var t=e.absId;return H[t]||(m(t),d(t,O))?void o.push(t):(i=0,!1)}),i){try{var r=c(o,{require:a.require,exports:a.exports,module:a}),s=a.factory,l="function"==typeof s?s.apply(e,r):s;null!=l&&(a.exports=l),a.invokeFactory=null,delete z[t]}catch(h){if(n=0,/^\[MODULE_MISS\]"([^"]+)/.test(h.message)){var p=a.depMkv[RegExp.$1];return void(p&&(p.hard=1))}throw h}u(t)}}}var n,a=A[t];a.invokeFactory=i,T(a.depPMs,function(e){p(e,function(){T(a.depRs,function(i){i.absId||i.mod!==e||(i.absId=v(i.id,t),y([i.absId],s))})})})}function d(e,t){return A[e]&&A[e].state>=t}function m(e){var t=A[e];t&&t.invokeFactory&&t.invokeFactory()}function c(e,t){var i=[];return T(e,function(e,n){i[n]=t[e]||V(e)}),i}function p(e,t){if(d(e,O))return void t();var i=N[e];i||(i=N[e]=[]),i.push(t)}function u(e){var t=N[e]||[],i=A[e];i.state=O;for(var n=t.length;n--;)t[n]();t.length=0,delete N[e]}function V(e){return d(e,O)?A[e].exports:null}function U(e){T(B,function(t){o(e,t.deps,t.factory)}),B.length=0,r(e)}function y(t,i,n,a){function o(){if(!l){var n=1;T(t,function(e){return H[e]?void 0:n=!!d(e,O)}),n&&(l=1,"function"==typeof i&&i.apply(e,c(t,H)))}}if("string"==typeof t){if(m(t),!d(t,O))throw new Error('[MODULE_MISS]"'+t+'" is not exists!');return V(t)}a=a||{};var l=0;t instanceof Array&&(o(),l||(T(t,function(e){H[e]||d(e,O)||(p(e,o),a[e]||(e.indexOf("!")>0?f:g)(e,n),r(e))}),s()))}function g(e){function t(){var t=i.readyState;if("undefined"==typeof t||/^(loaded|complete)$/.test(t)){i.onload=i.onreadystatechange=null,i=null,U(e);for(var n in z)r(n);s()}}if(!R[e]&&!A[e]){R[e]=1;var i=document.createElement("script");i.setAttribute("data-require-id",e),i.src=k(e+".js"),i.async=!0,i.readyState?i.onreadystatechange=t:i.onload=t,K(i)}}function f(e,t){function i(t){r.exports=t||!0,u(e)}function n(n){var r=t?A[t].require:P;n.load(o.res,r,i,a.call({id:e}))}if(!A[e]){var o=w(e),r={id:e,state:J};A[e]=r,i.fromText=function(e,t){z[e]=1,new Function(t)(),U(e)},n(V(o.mod))}}function b(e,t){var i=X(e,1,t);return i.sort(C),i}function _(){G.baseUrl=G.baseUrl.replace(/\/$/,"")+"/",Y=b(G.paths),q=b(G.map,1),T(q,function(e){e.v=b(e.v)}),Q=[],T(G.packages,function(e){var t=e;"string"==typeof e&&(t={name:e.split("/")[0],location:e,main:"main"}),t.location=t.location||t.name,t.main=(t.main||"main").replace(/\.js$/i,""),t.reg=S(t.name),Q.push(t)}),Q.sort(C),Z=b(G.urlArgs,1),j=b(G.noRequests),T(j,function(e){var t=e.v,i={};e.v=i,t instanceof Array||(t=[t]),T(t,function(e){i[e]=1})})}function x(e,t,i){T(t,function(t){return t.reg.test(e)?(i(t.v,t.k,t),!1):void 0})}function k(e){var t=/(\.[a-z0-9]+)$/i,i=/(\?[^#]*)$/,n="",a=e,o="";i.test(e)&&(o=RegExp.$1,e=e.replace(i,"")),t.test(e)&&(n=RegExp.$1,a=e.replace(t,""));var r,s=a;return x(a,Y,function(e,t){s=s.replace(t,e),r=1}),r||x(a,Q,function(e,t,i){s=s.replace(i.name,i.location)}),/^([a-z]{2,10}:\/)?\//i.test(s)||(s=G.baseUrl+s),s+=n+o,x(a,Z,function(e){s+=(s.indexOf("?")>0?"&":"?")+e}),s}function L(e){function t(t,n){if("string"==typeof t)return i[t]||(i[t]=y(v(t,e))),i[t];if(t instanceof Array){var a=[],o=[],r=[];T(t,function(t,i){var n=w(t),s=v(n.mod,e);o.push(s),z[s]=1,n.res?(a.push(s),r[i]=null):r[i]=s});var s={};T(o,function(e){var t;x(e,j,function(e){t=e}),t&&(t["*"]?s[e]=1:T(o,function(i){return t[i]?(s[e]=1,!1):void 0}))}),y(o,function(){T(r,function(i,n){null==i&&(r[n]=v(t[n],e))}),y(r,n,e)},e,s)}}var i={};return t.toUrl=function(t){return k(v(t,e))},t}function v(e,t){if(!e)return"";t=t||"";var i=w(e);if(!i)return e;var n=i.res,a=W(i.mod,t);if(T(Q,function(e){var t=e.name;return t===a?(a=t+"/"+e.main,!1):void 0}),x(t,q,function(e){x(a,e,function(e,t){a=a.replace(t,e)})}),n){var o=V(a);n=o.normalize?o.normalize(n,function(e){return v(e,t)}):v(n,t),a+="!"+n}return a}function W(e,t){if(0===e.indexOf(".")){var i=t.split("/"),n=e.split("/"),a=i.length-1,o=n.length,r=0,s=0;e:for(var l=0;o>l;l++){var h=n[l];switch(h){case"..":if(!(a>r))break e;r++,s++;break;case".":s++;break;default:break e}}return i.length=a-r,n=n.slice(s),i.concat(n).join("/")}return e}function w(e){var t=e.split("!");return t[0]?{mod:t[0],res:t[1]}:null}function X(e,t,i){var n=[];for(var a in e)if(e.hasOwnProperty(a)){var o={k:a,v:e[a]};n.push(o),t&&(o.reg="*"===a&&i?/^/:S(a))}return n}function I(){if($)return $;if(et&&"interactive"===et.readyState)return et;for(var e=document.getElementsByTagName("script"),t=e.length;t--;){var i=e[t];if("interactive"===i.readyState)return et=i,i}}function K(e){$=e,it?tt.insertBefore(e,it):tt.appendChild(e),$=null}function S(e){return new RegExp("^"+e+"(/|$)")}function T(e,t){if(e instanceof Array)for(var i=0,n=e.length;n>i&&t(e[i],i)!==!1;i++);}function C(e,t){var i=e.k||e.name,n=t.k||t.name;return"*"===n?-1:"*"===i?1:n.length-i.length}var E,A={},z={},M=1,J=2,F=3,O=4,P=L();t.version="1.8.6",t.loader="esl",t.toUrl=P.toUrl;var D;n.amd={};var N={},H={require:t,exports:1,module:1},B=[],R={},G={baseUrl:"./",paths:{},config:{},map:{},packages:[],waitSeconds:0,noRequests:{},urlArgs:{}};t.config=function(e){function t(e){a.push(e)}if(e){for(var i in G){var n=e[i],a=G[i];if(n)if("urlArgs"===i&&"string"==typeof n)G.urlArgs["*"]=n;else if(a instanceof Array)T(n,t);else if("object"==typeof a)for(var i in n)a[i]=n[i];else G[i]=n}_()}},_();var Y,Q,q,Z,j,$,et,tt=document.getElementsByTagName("head")[0],it=document.getElementsByTagName("base")[0];it&&(tt=it.parentNode),e.define||(e.define=n,e.require||(e.require=t),e.esl=t)}(this),define("echarts",["echarts/echarts"],function(e){return e}),define("echarts/echarts",["require","./config","zrender/tool/util","zrender/tool/event","zrender/tool/env","zrender","zrender/config","./chart/island","./component/toolbox","./component","./component/title","./component/tooltip","./component/legend","./util/ecData","./chart","zrender/tool/color","./component/timeline","zrender/shape/Image","zrender/loadingEffect/Bar","zrender/loadingEffect/Bubble","zrender/loadingEffect/DynamicLine","zrender/loadingEffect/Ring","zrender/loadingEffect/Spin","zrender/loadingEffect/Whirling","./theme/default"],function(e){function t(){r.Dispatcher.call(this)}function i(e){this._themeConfig=o.clone(a),this.dom=e,this._connected=!1,this._status={dragIn:!1,dragOut:!1,needRefresh:!1},this._curEventType=!1,this._chartList=[],this._messageCenter=new t,this._messageCenterOutSide=new t,this.resize=this.resize(),this._init()}function n(e,t,i,n,a){for(var o=e._chartList,r=o.length;r--;){var s=o[r];"function"==typeof s[t]&&s[t](i,n,a)}}var a=e("./config"),o=e("zrender/tool/util"),r=e("zrender/tool/event"),s={},l=e("zrender/tool/env").canvasSupported,h=new Date-0,d={},m="_echarts_instance_";s.version="2.1.10",s.dependencies={zrender:"2.0.6"},s.init=function(t,n){var a=e("zrender");(a.version||"1.0.3").replace(".","")-0r;r++){var l=p[r],h=c[l];o[h]="_on"+l.toLowerCase(),i.on(h,this._onzrevent)}this.chart={},this.component={};var d=e("./chart/island");this._island=new d(this._themeConfig,this._messageCenter,i,{},this),this.chart.island=this._island;var m=e("./component/toolbox");this._toolbox=new m(this._themeConfig,this._messageCenter,i,{},this),this.component.toolbox=this._toolbox;var u=e("./component");u.define("title",e("./component/title")),u.define("tooltip",e("./component/tooltip")),u.define("legend",e("./component/legend")),(0===i.getWidth()||0===i.getHeight())&&console.error("Dom’s width & height should be ready before init.")},__onevent:function(e){e.__echartsId=e.__echartsId||this.id;var t=e.__echartsId===this.id;switch(this._curEventType||(this._curEventType=e.type),e.type){case a.EVENT.LEGEND_SELECTED:this._onlegendSelected(e);break;case a.EVENT.DATA_ZOOM:if(!t){var i=this.component.dataZoom;i&&(i.silence(!0),i.absoluteZoom(e.zoom),i.silence(!1))}this._ondataZoom(e);break;case a.EVENT.DATA_RANGE:t&&this._ondataRange(e);break;case a.EVENT.MAGIC_TYPE_CHANGED:if(!t){var n=this.component.toolbox;n&&(n.silence(!0),n.setMagicType(e.magicType),n.silence(!1))}this._onmagicTypeChanged(e);break;case a.EVENT.DATA_VIEW_CHANGED:t&&this._ondataViewChanged(e);break;case a.EVENT.TOOLTIP_HOVER:t&&this._tooltipHover(e);break;case a.EVENT.RESTORE:this._onrestore();break;case a.EVENT.REFRESH:t&&this._onrefresh(e);break;case a.EVENT.TOOLTIP_IN_GRID:case a.EVENT.TOOLTIP_OUT_GRID:if(t){if(this._connected){var o=this.component.grid;o&&(e.x=(e.event.zrenderX-o.getX())/o.getWidth(),e.y=(e.event.zrenderY-o.getY())/o.getHeight())}}else{var o=this.component.grid;o&&this._zr.trigger("mousemove",{connectTrigger:!0,zrenderX:o.getX()+e.x*o.getWidth(),zrenderY:o.getY()+e.y*o.getHeight()})}}if(this._connected&&t&&this._curEventType===e.type){for(var r in this._connected)this._connected[r].connectedEventHandler(e);this._curEventType=null}(!t||!this._connected&&t)&&(this._curEventType=null)},_onclick:function(e){if(n(this,"onclick",e),e.target){var t=this._eventPackage(e.target);t&&null!=t.seriesIndex&&this._messageCenter.dispatch(a.EVENT.CLICK,e.event,t,this)}},_ondblclick:function(e){if(n(this,"ondblclick",e),e.target){var t=this._eventPackage(e.target);t&&null!=t.seriesIndex&&this._messageCenter.dispatch(a.EVENT.DBLCLICK,e.event,t,this)}},_onmouseover:function(e){if(e.target){var t=this._eventPackage(e.target);t&&null!=t.seriesIndex&&this._messageCenter.dispatch(a.EVENT.HOVER,e.event,t,this)}},_onmouseout:function(e){if(e.target){var t=this._eventPackage(e.target);t&&null!=t.seriesIndex&&this._messageCenter.dispatch(a.EVENT.MOUSEOUT,e.event,t,this)}},_ondragstart:function(e){this._status={dragIn:!1,dragOut:!1,needRefresh:!1},n(this,"ondragstart",e)},_ondragenter:function(e){n(this,"ondragenter",e)},_ondragover:function(e){n(this,"ondragover",e)},_ondragleave:function(e){n(this,"ondragleave",e)},_ondrop:function(e){n(this,"ondrop",e,this._status),this._island.ondrop(e,this._status)},_ondragend:function(e){if(n(this,"ondragend",e,this._status),this._timeline&&this._timeline.ondragend(e,this._status),this._island.ondragend(e,this._status),this._status.needRefresh){this._syncBackupData(this._option);var t=this._messageCenter;t.dispatch(a.EVENT.DATA_CHANGED,e.event,this._eventPackage(e.target),this),t.dispatch(a.EVENT.REFRESH,null,null,this)}},_onlegendSelected:function(e){this._status.needRefresh=!1,n(this,"onlegendSelected",e,this._status),this._status.needRefresh&&this._messageCenter.dispatch(a.EVENT.REFRESH,null,null,this)},_ondataZoom:function(e){this._status.needRefresh=!1,n(this,"ondataZoom",e,this._status),this._status.needRefresh&&this._messageCenter.dispatch(a.EVENT.REFRESH,null,null,this)},_ondataRange:function(e){this._clearEffect(),this._status.needRefresh=!1,n(this,"ondataRange",e,this._status),this._status.needRefresh&&this._zr.refresh()},_onmagicTypeChanged:function(){this._clearEffect(),this._render(this._toolbox.getMagicOption())},_ondataViewChanged:function(e){this._syncBackupData(e.option),this._messageCenter.dispatch(a.EVENT.DATA_CHANGED,null,e,this),this._messageCenter.dispatch(a.EVENT.REFRESH,null,null,this)},_tooltipHover:function(e){var t=[];n(this,"ontooltipHover",e,t)},_onrestore:function(){this.restore()},_onrefresh:function(e){this._refreshInside=!0,this.refresh(e),this._refreshInside=!1},_syncBackupData:function(e){this.component.dataZoom&&this.component.dataZoom.syncBackupData(e)},_eventPackage:function(t){if(t){var i=e("./util/ecData"),n=i.get(t,"seriesIndex"),a=i.get(t,"dataIndex");return a=-1!=n&&this.component.dataZoom?this.component.dataZoom.getRealDataIndex(n,a):a,{seriesIndex:n,seriesName:(i.get(t,"series")||{}).name,dataIndex:a,data:i.get(t,"data"),name:i.get(t,"name"),value:i.get(t,"value"),special:i.get(t,"special")}}},_render:function(t){this._mergeGlobalConifg(t);var i=t.backgroundColor;if(i)if(l||-1==i.indexOf("rgba"))this.dom.style.backgroundColor=i;else{var n=i.split(",");this.dom.style.filter="alpha(opacity="+100*n[3].substring(0,n[3].lastIndexOf(")"))+")",n.length=3,n[0]=n[0].replace("a",""),this.dom.style.backgroundColor=n.join(",")+")"}this._zr.clearAnimation(),this._chartList=[];var o=e("./chart"),r=e("./component");(t.xAxis||t.yAxis)&&(t.grid=t.grid||{},t.dataZoom=t.dataZoom||{});for(var s,h,d,m=["title","legend","tooltip","dataRange","roamController","grid","dataZoom","xAxis","yAxis","polar"],c=0,p=m.length;p>c;c++)h=m[c],d=this.component[h],t[h]?(d?d.refresh&&d.refresh(t):(s=r.get(/^[xy]Axis$/.test(h)?"axis":h),d=new s(this._themeConfig,this._messageCenter,this._zr,t,this,h),this.component[h]=d),this._chartList.push(d)):d&&(d.dispose(),this.component[h]=null,delete this.component[h]);for(var u,V,U,y={},c=0,p=t.series.length;p>c;c++)V=t.series[c].type,V?y[V]||(y[V]=!0,u=o.get(V),u?(this.chart[V]?(U=this.chart[V],U.refresh(t)):U=new u(this._themeConfig,this._messageCenter,this._zr,t,this),this._chartList.push(U),this.chart[V]=U):console.error(V+" has not been required.")):console.error("series["+c+"] chart type has not been defined.");for(V in this.chart)V==a.CHART_TYPE_ISLAND||y[V]||(this.chart[V].dispose(),this.chart[V]=null,delete this.chart[V]);this.component.grid&&this.component.grid.refixAxisShape(this.component),this._island.refresh(t),this._toolbox.refresh(t),t.animation&&!t.renderAsImage?this._zr.refresh():this._zr.render();var g="IMG"+this.id,f=document.getElementById(g);t.renderAsImage&&l?(f?f.src=this.getDataURL(t.renderAsImage):(f=this.getImage(t.renderAsImage),f.id=g,f.style.position="absolute",f.style.left=0,f.style.top=0,this.dom.firstChild.appendChild(f)),this.un(),this._zr.un(),this._disposeChartList(),this._zr.clear()):f&&f.parentNode.removeChild(f),f=null,this._option=t},restore:function(){this._clearEffect(),this._option=o.clone(this._optionRestore),this._disposeChartList(),this._island.clear(),this._toolbox.reset(this._option,!0),this._render(this._option)},refresh:function(e){this._clearEffect(),e=e||{};var t=e.option;!this._refreshInside&&t&&(t=this.getOption(),o.merge(t,e.option,!0),o.merge(this._optionRestore,e.option,!0),this._toolbox.reset(t)),this._island.refresh(t),this._toolbox.refresh(t),this._zr.clearAnimation();for(var i=0,n=this._chartList.length;n>i;i++)this._chartList[i].refresh&&this._chartList[i].refresh(t);this.component.grid&&this.component.grid.refixAxisShape(this.component),this._zr.refresh()},_disposeChartList:function(){this._clearEffect(),this._zr.clearAnimation();for(var e=this._chartList.length;e--;){var t=this._chartList[e];if(t){var i=t.type;this.chart[i]&&delete this.chart[i],this.component[i]&&delete this.component[i],t.dispose&&t.dispose()}}this._chartList=[]},_mergeGlobalConifg:function(t){for(var i=["backgroundColor","calculable","calculableColor","calculableHolderColor","nameConnector","valueConnector","animation","animationThreshold","animationDuration","animationEasing","addDataAnimation","symbolList","DRAG_ENABLE_TIME"],n=i.length;n--;){var a=i[n];null==t[a]&&(t[a]=this._themeConfig[a])}var o=t.color;o&&o.length||(o=this._themeConfig.color),l||(t.animation=!1,t.addDataAnimation=!1),this._zr.getColor=function(t){var i=e("zrender/tool/color");return i.getColor(t,o)}},setOption:function(e,t){return e.timeline?this._setTimelineOption(e):this._setOption(e,t)},_setOption:function(e,t){return this._option=!t&&this._option?o.merge(this.getOption(),o.clone(e),!0):o.clone(e),this._optionRestore=o.clone(this._option),this._option.series&&0!==this._option.series.length?(this.component.dataZoom&&(this._option.dataZoom||this._option.toolbox&&this._option.toolbox.feature&&this._option.toolbox.feature.dataZoom&&this._option.toolbox.feature.dataZoom.show)&&this.component.dataZoom.syncOption(this._option),this._toolbox.reset(this._option),this._render(this._option),this):void this._zr.clear()},getOption:function(){function e(e){var n=i._optionRestore[e];if(n)if(n instanceof Array)for(var a=n.length;a--;)t[e][a].data=o.clone(n[a].data);else t[e].data=o.clone(n.data)}var t=o.clone(this._option),i=this;return e("xAxis"),e("yAxis"),e("series"),t},setSeries:function(e,t){return t?(this._option.series=e,this.setOption(this._option,t)):this.setOption({series:e}),this},getSeries:function(){return this.getOption().series},_setTimelineOption:function(t){this._timeline&&this._timeline.dispose();var i=e("./component/timeline"),n=new i(this._themeConfig,this._messageCenter,this._zr,t,this);return this._timeline=n,this.component.timeline=this._timeline,this},addData:function(e,t,i,n,r){for(var s=e instanceof Array?e:[[e,t,i,n,r]],l=this.getOption(),h=this._optionRestore,d=0,m=s.length;m>d;d++){e=s[d][0],t=s[d][1],i=s[d][2],n=s[d][3],r=s[d][4];var c=h.series[e],p=i?"unshift":"push",u=i?"pop":"shift";if(c){var V=c.data,U=l.series[e].data;if(V[p](t),U[p](t),n||(V[u](),t=U[u]()),null!=r){var y,g;if(c.type===a.CHART_TYPE_PIE&&(y=h.legend)&&(g=y.data)){var f=l.legend.data;if(g[p](r),f[p](r),!n){var b=o.indexOf(g,t.name);-1!=b&&g.splice(b,1),b=o.indexOf(f,t.name),-1!=b&&f.splice(b,1)}}else if(null!=h.xAxis&&null!=h.yAxis){var _,x,k=c.xAxisIndex||0;(null==h.xAxis[k].type||"category"===h.xAxis[k].type)&&(_=h.xAxis[k].data,x=l.xAxis[k].data,_[p](r),x[p](r),n||(_[u](),x[u]())),k=c.yAxisIndex||0,"category"===h.yAxis[k].type&&(_=h.yAxis[k].data,x=l.yAxis[k].data,_[p](r),x[p](r),n||(_[u](),x[u]()))}}this._option.series[e].data=l.series[e].data}}this._zr.clearAnimation();for(var L=this._chartList,d=0,m=L.length;m>d;d++)l.addDataAnimation&&L[d].addDataAnimation&&L[d].addDataAnimation(s);this.component.dataZoom&&this.component.dataZoom.syncOption(l),this._option=l;var v=this;return setTimeout(function(){if(v._zr){v._zr.clearAnimation();for(var e=0,t=L.length;t>e;e++)L[e].motionlessOnce=l.addDataAnimation&&L[e].addDataAnimation;v._messageCenter.dispatch(a.EVENT.REFRESH,null,{option:l},v)}},l.addDataAnimation?500:0),this},addMarkPoint:function(e,t){return this._addMark(e,t,"markPoint")},addMarkLine:function(e,t){return this._addMark(e,t,"markLine")},_addMark:function(e,t,i){var n,a=this._option.series;if(a&&(n=a[e])){var r=this._optionRestore.series,s=r[e],l=n[i],h=s[i];l=n[i]=l||{data:[]},h=s[i]=h||{data:[]};for(var d in t)"data"===d?(l.data=l.data.concat(t.data),h.data=h.data.concat(t.data)):"object"!=typeof t[d]||null==l[d]?l[d]=h[d]=t[d]:(o.merge(l[d],t[d],!0),o.merge(h[d],t[d],!0));var m=this.chart[n.type];m&&m.addMark(e,t,i)}return this},delMarkPoint:function(e,t){return this._delMark(e,t,"markPoint")},delMarkLine:function(e,t){return this._delMark(e,t,"markLine")},_delMark:function(e,t,i){var n,a,o,r=this._option.series;if(!(r&&(n=r[e])&&(a=n[i])&&(o=a.data)))return this;t=t.split(" > ");for(var s=-1,l=0,h=o.length;h>l;l++){var d=o[l];if(d instanceof Array){if(d[0].name===t[0]&&d[1].name===t[1]){s=l;break}}else if(d.name===t[0]){s=l;break}}if(s>-1){o.splice(s,1),this._optionRestore.series[e][i].data.splice(s,1);var m=this.chart[n.type];m&&m.delMark(e,t.join(" > "),i)}return this},getDom:function(){return this.dom},getZrender:function(){return this._zr},getDataURL:function(e){if(!l)return"";if(0===this._chartList.length){var t="IMG"+this.id,i=document.getElementById(t);if(i)return i.src}var n=this.component.tooltip;switch(n&&n.hideTip(),e){case"jpeg":break;default:e="png"}var a=this._option.backgroundColor;return a&&"rgba(0,0,0,0)"===a.replace(" ","")&&(a="#fff"),this._zr.toDataURL("image/"+e,a)},getImage:function(e){var t=this._optionRestore.title,i=document.createElement("img");return i.src=this.getDataURL(e),i.title=t&&t.text||"ECharts",i},getConnectedDataURL:function(t){if(!this.isConnected())return this.getDataURL(t);var i=this.dom,n={self:{img:this.getDataURL(t),left:i.offsetLeft,top:i.offsetTop,right:i.offsetLeft+i.offsetWidth,bottom:i.offsetTop+i.offsetHeight}},a=n.self.left,o=n.self.top,r=n.self.right,s=n.self.bottom;for(var l in this._connected)i=this._connected[l].getDom(),n[l]={img:this._connected[l].getDataURL(t),left:i.offsetLeft,top:i.offsetTop,right:i.offsetLeft+i.offsetWidth,bottom:i.offsetTop+i.offsetHeight},a=Math.min(a,n[l].left),o=Math.min(o,n[l].top),r=Math.max(r,n[l].right),s=Math.max(s,n[l].bottom);var h=document.createElement("div");h.style.position="absolute",h.style.left="-4000px",h.style.width=r-a+"px",h.style.height=s-o+"px",document.body.appendChild(h);var d=e("zrender").init(h),m=e("zrender/shape/Image");for(var l in n)d.addShape(new m({style:{x:n[l].left-a,y:n[l].top-o,image:n[l].img}}));d.render();var c=this._option.backgroundColor;c&&"rgba(0,0,0,0)"===c.replace(/ /g,"")&&(c="#fff");var p=d.toDataURL("image/png",c);return setTimeout(function(){d.dispose(),h.parentNode.removeChild(h),h=null},100),p},getConnectedImage:function(e){var t=this._optionRestore.title,i=document.createElement("img");return i.src=this.getConnectedDataURL(e),i.title=t&&t.text||"ECharts",i},on:function(e,t){return this._messageCenterOutSide.bind(e,t,this),this},un:function(e,t){return this._messageCenterOutSide.unbind(e,t),this},connect:function(e){if(!e)return this;if(this._connected||(this._connected={}),e instanceof Array)for(var t=0,i=e.length;i>t;t++)this._connected[e[t].id]=e[t];else this._connected[e.id]=e;return this},disConnect:function(e){if(!e||!this._connected)return this;if(e instanceof Array)for(var t=0,i=e.length;i>t;t++)delete this._connected[e[t].id];else delete this._connected[e.id];for(var n in this._connected)return this;return this._connected=!1,this},connectedEventHandler:function(e){e.__echartsId!=this.id&&this._onevent(e)},isConnected:function(){return!!this._connected},showLoading:function(t){var i={bar:e("zrender/loadingEffect/Bar"),bubble:e("zrender/loadingEffect/Bubble"),dynamicLine:e("zrender/loadingEffect/DynamicLine"),ring:e("zrender/loadingEffect/Ring"),spin:e("zrender/loadingEffect/Spin"),whirling:e("zrender/loadingEffect/Whirling")};this._toolbox.hideDataView(),t=t||{};var n=t.textStyle||{};t.textStyle=n;var a=o.merge(o.clone(n),this._themeConfig.textStyle);n.textFont=a.fontStyle+" "+a.fontWeight+" "+a.fontSize+"px "+a.fontFamily,n.text=t.text||this._themeConfig.loadingText,null!=t.x&&(n.x=t.x),null!=t.y&&(n.y=t.y),t.effectOption=t.effectOption||{},t.effectOption.textStyle=n;var r=t.effect;return("string"==typeof r||null==r)&&(r=i[t.effect||"spin"]),this._zr.showLoading(new r(t.effectOption)),this},hideLoading:function(){return this._zr.hideLoading(),this},setTheme:function(t){if(t){if("string"==typeof t)switch(t){default:t=e("./theme/default")}else t=t||{};for(var i in this._themeConfig)delete this._themeConfig[i];for(var i in a)this._themeConfig[i]=o.clone(a[i]);t.color&&(this._themeConfig.color=[]),t.symbolList&&(this._themeConfig.symbolList=[]),o.merge(this._themeConfig,o.clone(t),!0)}l||(this._themeConfig.textStyle.fontFamily=this._themeConfig.textStyle.fontFamily2),this._timeline&&this._timeline.setTheme(!0),this._optionRestore&&this.restore()},resize:function(){var e=this;return function(){if(e._clearEffect(),e._zr.resize(),e._option&&e._option.renderAsImage&&l)return e._render(e._option),e;e._zr.clearAnimation(),e._island.resize(),e._toolbox.resize(),e._timeline&&e._timeline.resize();for(var t=0,i=e._chartList.length;i>t;t++)e._chartList[t].resize&&e._chartList[t].resize();return e.component.grid&&e.component.grid.refixAxisShape(e.component),e._zr.refresh(),e._messageCenter.dispatch(a.EVENT.RESIZE,null,null,e),e}},_clearEffect:function(){this._zr.modLayer(a.EFFECT_ZLEVEL,{motionBlur:!1}),this._zr.painter.clearLayer(a.EFFECT_ZLEVEL)},clear:function(){return this._disposeChartList(),this._zr.clear(),this._option={},this._optionRestore={},this.dom.style.backgroundColor=null,this},dispose:function(){var e=this.dom.getAttribute(m);e&&delete d[e],this._island.dispose(),this._toolbox.dispose(),this._timeline&&this._timeline.dispose(),this._messageCenter.unbind(),this.clear(),this._zr.dispose(),this._zr=null}},s}),define("echarts/config",[],function(){var e={CHART_TYPE_LINE:"line",CHART_TYPE_BAR:"bar",CHART_TYPE_SCATTER:"scatter",CHART_TYPE_PIE:"pie",CHART_TYPE_RADAR:"radar",CHART_TYPE_MAP:"map",CHART_TYPE_K:"k",CHART_TYPE_ISLAND:"island",CHART_TYPE_FORCE:"force",CHART_TYPE_CHORD:"chord",CHART_TYPE_GAUGE:"gauge",CHART_TYPE_FUNNEL:"funnel",CHART_TYPE_EVENTRIVER:"eventRiver",COMPONENT_TYPE_TITLE:"title",COMPONENT_TYPE_LEGEND:"legend",COMPONENT_TYPE_DATARANGE:"dataRange",COMPONENT_TYPE_DATAVIEW:"dataView",COMPONENT_TYPE_DATAZOOM:"dataZoom",COMPONENT_TYPE_TOOLBOX:"toolbox",COMPONENT_TYPE_TOOLTIP:"tooltip",COMPONENT_TYPE_GRID:"grid",COMPONENT_TYPE_AXIS:"axis",COMPONENT_TYPE_POLAR:"polar",COMPONENT_TYPE_X_AXIS:"xAxis",COMPONENT_TYPE_Y_AXIS:"yAxis",COMPONENT_TYPE_AXIS_CATEGORY:"categoryAxis",COMPONENT_TYPE_AXIS_VALUE:"valueAxis",COMPONENT_TYPE_TIMELINE:"timeline",COMPONENT_TYPE_ROAMCONTROLLER:"roamController",backgroundColor:"rgba(0,0,0,0)",color:["#ff7f50","#87cefa","#da70d6","#32cd32","#6495ed","#ff69b4","#ba55d3","#cd5c5c","#ffa500","#40e0d0","#1e90ff","#ff6347","#7b68ee","#00fa9a","#ffd700","#6699FF","#ff6666","#3cb371","#b8860b","#30e0e0"],title:{text:"",subtext:"",x:"left",y:"top",backgroundColor:"rgba(0,0,0,0)",borderColor:"#ccc",borderWidth:0,padding:5,itemGap:5,textStyle:{fontSize:18,fontWeight:"bolder",color:"#333"},subtextStyle:{color:"#aaa"}},legend:{show:!0,orient:"horizontal",x:"center",y:"top",backgroundColor:"rgba(0,0,0,0)",borderColor:"#ccc",borderWidth:0,padding:5,itemGap:10,itemWidth:20,itemHeight:14,textStyle:{color:"#333"},selectedMode:!0},dataRange:{show:!0,orient:"vertical",x:"left",y:"bottom",backgroundColor:"rgba(0,0,0,0)",borderColor:"#ccc",borderWidth:0,padding:5,itemGap:10,itemWidth:20,itemHeight:14,precision:0,splitNumber:5,calculable:!1,hoverLink:!0,realtime:!0,color:["#006edd","#e0ffff"],textStyle:{color:"#333"}},toolbox:{show:!1,orient:"horizontal",x:"right",y:"top",color:["#1e90ff","#22bb22","#4b0082","#d2691e"],disableColor:"#ddd",effectiveColor:"red",backgroundColor:"rgba(0,0,0,0)",borderColor:"#ccc",borderWidth:0,padding:5,itemGap:10,itemSize:16,showTitle:!0,feature:{mark:{show:!1,title:{mark:"Auxiliary line",markUndo:"Undo",markClear:"Clear"},lineStyle:{width:1,color:"#1e90ff",type:"dashed"}},dataZoom:{show:!1,title:{dataZoom:"Zoom",dataZoomReset:"Zoom/Reset"}},dataView:{show:!1,title:"Data View",readOnly:!1,lang:["View","Close","Refresh"]},magicType:{show:!1,title:{line:"Line",bar:"Bar",stack:"Stack",tiled:"Tiled",force:"Force",chord:"Chord",pie:"Pie",funnel:"Funnel"},type:[]},restore:{show:!1,title:"Restore"},saveAsImage:{show:!1,title:"Save As Image",type:"png",lang:["Save As Image"]}}},tooltip:{show:!0,showContent:!0,trigger:"item",islandFormatter:"{a}
{b} : {c}",showDelay:20,hideDelay:100,transitionDuration:.4,enterable:!1,backgroundColor:"rgba(0,0,0,0.7)",borderColor:"#333",borderRadius:4,borderWidth:0,padding:5,axisPointer:{type:"line",lineStyle:{color:"#48b",width:2,type:"solid"},crossStyle:{color:"#1e90ff",width:1,type:"dashed"},shadowStyle:{color:"rgba(150,150,150,0.3)",width:"auto",type:"default"}},textStyle:{color:"#fff"}},dataZoom:{show:!1,orient:"horizontal",backgroundColor:"rgba(0,0,0,0)",dataBackgroundColor:"#eee",fillerColor:"rgba(144,197,237,0.2)",handleColor:"rgba(70,130,180,0.8)",showDetail:!0,realtime:!0},grid:{x:80,y:60,x2:80,y2:60,backgroundColor:"rgba(0,0,0,0)",borderWidth:1,borderColor:"#ccc"},categoryAxis:{show:!0,position:"bottom",name:"",nameLocation:"end",nameTextStyle:{},boundaryGap:!0,axisLine:{show:!0,onZero:!0,lineStyle:{color:"#48b",width:2,type:"solid"}},axisTick:{show:!0,interval:"auto",inside:!1,length:5,lineStyle:{color:"#333",width:1}},axisLabel:{show:!0,interval:"auto",rotate:0,margin:8,textStyle:{color:"#333"}},splitLine:{show:!0,lineStyle:{color:["#ccc"],width:1,type:"solid"}},splitArea:{show:!1,areaStyle:{color:["rgba(250,250,250,0.3)","rgba(200,200,200,0.3)"]}}},valueAxis:{show:!0,position:"left",name:"",nameLocation:"end",nameTextStyle:{},boundaryGap:[0,0],axisLine:{show:!0,onZero:!0,lineStyle:{color:"#48b",width:2,type:"solid"}},axisTick:{show:!1,inside:!1,length:5,lineStyle:{color:"#333",width:1}},axisLabel:{show:!0,rotate:0,margin:8,textStyle:{color:"#333"}},splitLine:{show:!0,lineStyle:{color:["#ccc"],width:1,type:"solid"}},splitArea:{show:!1,areaStyle:{color:["rgba(250,250,250,0.3)","rgba(200,200,200,0.3)"]}}},polar:{center:["50%","50%"],radius:"75%",startAngle:90,boundaryGap:[0,0],splitNumber:5,name:{show:!0,textStyle:{color:"#333"}},axisLine:{show:!0,lineStyle:{color:"#ccc",width:1,type:"solid"}},axisLabel:{show:!1,textStyle:{color:"#333"}},splitArea:{show:!0,areaStyle:{color:["rgba(250,250,250,0.3)","rgba(200,200,200,0.3)"]}},splitLine:{show:!0,lineStyle:{width:1,color:"#ccc"}},type:"polygon"},timeline:{show:!0,type:"time",notMerge:!1,realtime:!0,x:80,x2:80,y2:0,height:50,backgroundColor:"rgba(0,0,0,0)",borderColor:"#ccc",borderWidth:0,padding:5,controlPosition:"left",autoPlay:!1,loop:!0,playInterval:2e3,lineStyle:{width:1,color:"#666",type:"dashed"},label:{show:!0,interval:"auto",rotate:0,textStyle:{color:"#333"}},checkpointStyle:{symbol:"auto",symbolSize:"auto",color:"auto",borderColor:"auto",borderWidth:"auto",label:{show:!1,textStyle:{color:"auto"}}},controlStyle:{normal:{color:"#333"},emphasis:{color:"#1e90ff"}},symbol:"emptyDiamond",symbolSize:4,currentIndex:0},roamController:{show:!0,x:"left",y:"top",width:80,height:120,backgroundColor:"rgba(0,0,0,0)",borderColor:"#ccc",borderWidth:0,padding:5,handleColor:"#6495ed",fillerColor:"#fff",step:15,mapTypeControl:null},bar:{clickable:!0,legendHoverLink:!0,xAxisIndex:0,yAxisIndex:0,barMinHeight:0,barGap:"30%",barCategoryGap:"20%",itemStyle:{normal:{barBorderColor:"#fff",barBorderRadius:0,barBorderWidth:0,label:{show:!1}},emphasis:{barBorderColor:"#fff",barBorderRadius:0,barBorderWidth:0,label:{show:!1}}}},line:{clickable:!0,legendHoverLink:!0,xAxisIndex:0,yAxisIndex:0,itemStyle:{normal:{label:{show:!1},lineStyle:{width:2,type:"solid",shadowColor:"rgba(0,0,0,0)",shadowBlur:0,shadowOffsetX:0,shadowOffsetY:0}},emphasis:{label:{show:!1}}},symbolSize:2,showAllSymbol:!1},k:{clickable:!0,legendHoverLink:!1,xAxisIndex:0,yAxisIndex:0,itemStyle:{normal:{color:"#fff",color0:"#00aa11",lineStyle:{width:1,color:"#ff3200",color0:"#00aa11"}},emphasis:{}}},scatter:{clickable:!0,legendHoverLink:!0,xAxisIndex:0,yAxisIndex:0,symbolSize:4,large:!1,largeThreshold:2e3,itemStyle:{normal:{label:{show:!1,formatter:function(e,t,i){return"undefined"!=typeof i[2]?i[2]:i[0]+" , "+i[1] +}}},emphasis:{label:{show:!1,formatter:function(e,t,i){return"undefined"!=typeof i[2]?i[2]:i[0]+" , "+i[1]}}}}},radar:{clickable:!0,legendHoverLink:!0,polarIndex:0,itemStyle:{normal:{label:{show:!1},lineStyle:{width:2,type:"solid"}},emphasis:{label:{show:!1}}},symbolSize:2},pie:{clickable:!0,legendHoverLink:!0,center:["50%","50%"],radius:[0,"75%"],clockWise:!0,startAngle:90,minAngle:0,selectedOffset:10,itemStyle:{normal:{borderColor:"rgba(0,0,0,0)",borderWidth:1,label:{show:!0,position:"outer"},labelLine:{show:!0,length:20,lineStyle:{width:1,type:"solid"}}},emphasis:{borderColor:"rgba(0,0,0,0)",borderWidth:1,label:{show:!1},labelLine:{show:!1,length:20,lineStyle:{width:1,type:"solid"}}}}},map:{mapType:"china",mapValuePrecision:0,showLegendSymbol:!0,dataRangeHoverLink:!0,hoverable:!0,clickable:!0,itemStyle:{normal:{borderColor:"rgba(0,0,0,0)",borderWidth:1,areaStyle:{color:"#ccc"},label:{show:!1,textStyle:{color:"rgb(139,69,19)"}}},emphasis:{borderColor:"rgba(0,0,0,0)",borderWidth:1,areaStyle:{color:"rgba(255,215,0,0.8)"},label:{show:!1,textStyle:{color:"rgb(100,0,0)"}}}}},force:{center:["50%","50%"],size:"100%",preventOverlap:!1,coolDown:.99,minRadius:10,maxRadius:20,ratioScaling:!1,large:!1,useWorker:!1,steps:1,scaling:1,gravity:1,symbol:"circle",symbolSize:0,linkSymbol:null,linkSymbolSize:[10,15],draggable:!0,clickable:!0,roam:!1,itemStyle:{normal:{label:{show:!1,position:"inside"},nodeStyle:{brushType:"both",borderColor:"#5182ab",borderWidth:1},linkStyle:{color:"#5182ab",width:1,type:"line"}},emphasis:{label:{show:!1},nodeStyle:{},linkStyle:{opacity:0}}}},chord:{clickable:!0,radius:["65%","75%"],center:["50%","50%"],padding:2,sort:"none",sortSub:"none",startAngle:90,clockWise:!0,ribbonType:!0,minRadius:10,maxRadius:20,symbol:"circle",showScale:!1,showScaleText:!1,itemStyle:{normal:{borderWidth:0,borderColor:"#000",label:{show:!0,rotate:!1,distance:5},chordStyle:{width:1,color:"black",borderWidth:1,borderColor:"#999",opacity:.5}},emphasis:{borderWidth:0,borderColor:"#000",chordStyle:{width:1,color:"black",borderWidth:1,borderColor:"#999"}}}},gauge:{center:["50%","50%"],legendHoverLink:!0,radius:"75%",startAngle:225,endAngle:-45,min:0,max:100,precision:0,splitNumber:10,axisLine:{show:!0,lineStyle:{color:[[.2,"#228b22"],[.8,"#48b"],[1,"#ff4500"]],width:30}},axisTick:{show:!0,splitNumber:5,length:8,lineStyle:{color:"#eee",width:1,type:"solid"}},axisLabel:{show:!0,textStyle:{color:"auto"}},splitLine:{show:!0,length:30,lineStyle:{color:"#eee",width:2,type:"solid"}},pointer:{show:!0,length:"80%",width:8,color:"auto"},title:{show:!0,offsetCenter:[0,"-40%"],textStyle:{color:"#333",fontSize:15}},detail:{show:!0,backgroundColor:"rgba(0,0,0,0)",borderWidth:0,borderColor:"#ccc",width:100,height:40,offsetCenter:[0,"40%"],textStyle:{color:"auto",fontSize:30}}},funnel:{clickable:!0,legendHoverLink:!0,x:80,y:60,x2:80,y2:60,min:0,max:100,minSize:"0%",maxSize:"100%",sort:"descending",gap:0,funnelAlign:"center",itemStyle:{normal:{borderColor:"#fff",borderWidth:1,label:{show:!0,position:"outer"},labelLine:{show:!0,length:10,lineStyle:{width:1,type:"solid"}}},emphasis:{borderColor:"rgba(0,0,0,0)",borderWidth:1,label:{show:!0},labelLine:{show:!0}}}},eventRiver:{clickable:!0,legendHoverLink:!0,itemStyle:{normal:{borderColor:"rgba(0,0,0,0)",borderWidth:1,label:{show:!0,position:"inside",formatter:"{b}"}},emphasis:{borderColor:"rgba(0,0,0,0)",borderWidth:1,label:{show:!0}}}},island:{r:15,calculateStep:.1},markPoint:{clickable:!0,symbol:"pin",symbolSize:10,large:!1,effect:{show:!1,loop:!0,period:15,scaleSize:2},itemStyle:{normal:{borderWidth:2,label:{show:!0,position:"inside"}},emphasis:{label:{show:!0}}}},markLine:{clickable:!0,symbol:["circle","arrow"],symbolSize:[2,4],large:!1,effect:{show:!1,loop:!0,period:15,scaleSize:2},itemStyle:{normal:{borderWidth:1.5,label:{show:!0,position:"end"},lineStyle:{type:"dashed"}},emphasis:{label:{show:!1},lineStyle:{}}}},textStyle:{decoration:"none",fontFamily:"Arial, Verdana, sans-serif",fontFamily2:"微软雅黑",fontSize:12,fontStyle:"normal",fontWeight:"normal"},EVENT:{REFRESH:"refresh",RESTORE:"restore",RESIZE:"resize",CLICK:"click",DBLCLICK:"dblclick",HOVER:"hover",MOUSEOUT:"mouseout",DATA_CHANGED:"dataChanged",DATA_ZOOM:"dataZoom",DATA_RANGE:"dataRange",DATA_RANGE_HOVERLINK:"dataRangeHoverLink",LEGEND_SELECTED:"legendSelected",LEGEND_HOVERLINK:"legendHoverLink",MAP_SELECTED:"mapSelected",PIE_SELECTED:"pieSelected",MAGIC_TYPE_CHANGED:"magicTypeChanged",DATA_VIEW_CHANGED:"dataViewChanged",TIMELINE_CHANGED:"timelineChanged",MAP_ROAM:"mapRoam",FORCE_LAYOUT_END:"forceLayoutEnd",TOOLTIP_HOVER:"tooltipHover",TOOLTIP_IN_GRID:"tooltipInGrid",TOOLTIP_OUT_GRID:"tooltipOutGrid",ROAMCONTROLLER:"roamController"},DRAG_ENABLE_TIME:120,EFFECT_ZLEVEL:7,symbolList:["circle","rectangle","triangle","diamond","emptyCircle","emptyRectangle","emptyTriangle","emptyDiamond"],loadingText:"Loading...",calculable:!1,calculableColor:"rgba(255,165,0,0.6)",calculableHolderColor:"#ccc",nameConnector:" & ",valueConnector:": ",animation:!0,addDataAnimation:!0,animationThreshold:2e3,animationDuration:2e3,animationEasing:"ExponentialOut"};return e}),define("zrender/tool/util",["require","../dep/excanvas"],function(e){function t(e){if("object"==typeof e&&null!==e){var i=e;if(e instanceof Array){i=[];for(var n=0,a=e.length;a>n;n++)i[n]=t(e[n])}else if(!V[Object.prototype.toString.call(e)]){i={};for(var o in e)e.hasOwnProperty(o)&&(i[o]=t(e[o]))}return i}return e}function i(e,t,i,a){t.hasOwnProperty(i)&&("object"!=typeof e[i]||V[Object.prototype.toString.call(e[i])]?!a&&i in e||(e[i]=t[i]):n(e[i],t[i],a))}function n(e,t,n){for(var a in t)i(e,t,a,n);return e}function a(){if(!d)if(e("../dep/excanvas"),window.G_vmlCanvasManager){var t=document.createElement("div");t.style.position="absolute",t.style.top="-1000px",document.body.appendChild(t),d=G_vmlCanvasManager.initElement(t).getContext("2d")}else d=document.createElement("canvas").getContext("2d");return d}function o(){return c||(m=document.createElement("canvas"),p=m.width,u=m.height,c=m.getContext("2d")),c}function r(e,t){var i,n=100;e+U>p&&(p=e+U+n,m.width=p,i=!0),t+y>u&&(u=t+y+n,m.height=u,i=!0),-U>e&&(U=Math.ceil(-e/n)*n,p+=U,m.width=p,i=!0),-y>t&&(y=Math.ceil(-t/n)*n,u+=y,m.height=u,i=!0),i&&c.translate(U,y)}function s(){return{x:U,y:y}}function l(e,t){if(e.indexOf)return e.indexOf(t);for(var i=0,n=e.length;n>i;i++)if(e[i]===t)return i;return-1}function h(e,t){function i(){}var n=e.prototype;i.prototype=t.prototype,e.prototype=new i;for(var a in n)e.prototype[a]=n[a];e.constructor=e}var d,m,c,p,u,V={"[object Function]":1,"[object RegExp]":1,"[object Date]":1,"[object Error]":1,"[object CanvasGradient]":1},U=0,y=0;return{inherits:h,clone:t,merge:n,getContext:a,getPixelContext:o,getPixelOffset:s,adjustCanvasSize:r,indexOf:l}}),define("zrender/tool/event",["require","../mixin/Eventful"],function(e){"use strict";function t(e){return"undefined"!=typeof e.zrenderX&&e.zrenderX||"undefined"!=typeof e.offsetX&&e.offsetX||"undefined"!=typeof e.layerX&&e.layerX||"undefined"!=typeof e.clientX&&e.clientX}function i(e){return"undefined"!=typeof e.zrenderY&&e.zrenderY||"undefined"!=typeof e.offsetY&&e.offsetY||"undefined"!=typeof e.layerY&&e.layerY||"undefined"!=typeof e.clientY&&e.clientY}function n(e){return"undefined"!=typeof e.zrenderDelta&&e.zrenderDelta||"undefined"!=typeof e.wheelDelta&&e.wheelDelta||"undefined"!=typeof e.detail&&-e.detail}var a=e("../mixin/Eventful"),o="function"==typeof window.addEventListener?function(e){e.preventDefault(),e.stopPropagation(),e.cancelBubble=!0}:function(e){e.returnValue=!1,e.cancelBubble=!0};return{getX:t,getY:i,getDelta:n,stop:o,Dispatcher:a}}),define("zrender/tool/env",[],function(){function e(e){var t=this.os={},i=this.browser={},n=e.match(/Web[kK]it[\/]{0,1}([\d.]+)/),a=e.match(/(Android);?[\s\/]+([\d.]+)?/),o=e.match(/(iPad).*OS\s([\d_]+)/),r=e.match(/(iPod)(.*OS\s([\d_]+))?/),s=!o&&e.match(/(iPhone\sOS)\s([\d_]+)/),l=e.match(/(webOS|hpwOS)[\s\/]([\d.]+)/),h=l&&e.match(/TouchPad/),d=e.match(/Kindle\/([\d.]+)/),m=e.match(/Silk\/([\d._]+)/),c=e.match(/(BlackBerry).*Version\/([\d.]+)/),p=e.match(/(BB10).*Version\/([\d.]+)/),u=e.match(/(RIM\sTablet\sOS)\s([\d.]+)/),V=e.match(/PlayBook/),U=e.match(/Chrome\/([\d.]+)/)||e.match(/CriOS\/([\d.]+)/),y=e.match(/Firefox\/([\d.]+)/),g=e.match(/MSIE ([\d.]+)/),f=n&&e.match(/Mobile\//)&&!U,b=e.match(/(iPhone|iPod|iPad).*AppleWebKit(?!.*Safari)/)&&!U,g=e.match(/MSIE\s([\d.]+)/);return(i.webkit=!!n)&&(i.version=n[1]),a&&(t.android=!0,t.version=a[2]),s&&!r&&(t.ios=t.iphone=!0,t.version=s[2].replace(/_/g,".")),o&&(t.ios=t.ipad=!0,t.version=o[2].replace(/_/g,".")),r&&(t.ios=t.ipod=!0,t.version=r[3]?r[3].replace(/_/g,"."):null),l&&(t.webos=!0,t.version=l[2]),h&&(t.touchpad=!0),c&&(t.blackberry=!0,t.version=c[2]),p&&(t.bb10=!0,t.version=p[2]),u&&(t.rimtabletos=!0,t.version=u[2]),V&&(i.playbook=!0),d&&(t.kindle=!0,t.version=d[1]),m&&(i.silk=!0,i.version=m[1]),!m&&t.android&&e.match(/Kindle Fire/)&&(i.silk=!0),U&&(i.chrome=!0,i.version=U[1]),y&&(i.firefox=!0,i.version=y[1]),g&&(i.ie=!0,i.version=g[1]),f&&(e.match(/Safari/)||t.ios)&&(i.safari=!0),b&&(i.webview=!0),g&&(i.ie=!0,i.version=g[1]),t.tablet=!!(o||V||a&&!e.match(/Mobile/)||y&&e.match(/Tablet/)||g&&!e.match(/Phone/)&&e.match(/Touch/)),t.phone=!(t.tablet||t.ipod||!(a||s||l||c||p||U&&e.match(/Android/)||U&&e.match(/CriOS\/([\d.]+)/)||y&&e.match(/Mobile/)||g&&e.match(/Touch/))),{browser:i,os:t,canvasSupported:document.createElement("canvas").getContext?!0:!1}}return e(navigator.userAgent)}),define("zrender",["zrender/zrender"],function(e){return e}),define("zrender/zrender",["require","./dep/excanvas","./tool/util","./tool/log","./tool/guid","./Handler","./Painter","./Storage","./animation/Animation","./tool/env"],function(e){function t(e){return function(){for(var t=e.animatingElements,i=0,n=t.length;n>i;i++)e.storage.mod(t[i].id);(t.length||e._needsRefreshNextFrame)&&e.refresh()}}e("./dep/excanvas");var i=e("./tool/util"),n=e("./tool/log"),a=e("./tool/guid"),o=e("./Handler"),r=e("./Painter"),s=e("./Storage"),l=e("./animation/Animation"),h={},d={};d.version="2.0.6",d.init=function(e){var t=new m(a(),e);return h[t.id]=t,t},d.dispose=function(e){if(e)e.dispose();else{for(var t in h)h[t].dispose();h={}}return d},d.getInstance=function(e){return h[e]},d.delInstance=function(e){return delete h[e],d};var m=function(i,n){this.id=i,this.env=e("./tool/env"),this.storage=new s,this.painter=new r(n,this.storage),this.handler=new o(n,this.storage,this.painter),this.animatingElements=[],this.animation=new l({stage:{update:t(this)}}),this.animation.start();var a=this;this.painter.refreshNextFrame=function(){a.refreshNextFrame()},this._needsRefreshNextFrame=!1};return m.prototype.getId=function(){return this.id},m.prototype.addShape=function(e){return this.storage.addRoot(e),this},m.prototype.addGroup=function(e){return this.storage.addRoot(e),this},m.prototype.delShape=function(e){return this.storage.delRoot(e),this},m.prototype.delGroup=function(e){return this.storage.delRoot(e),this},m.prototype.modShape=function(e,t){return this.storage.mod(e,t),this},m.prototype.modGroup=function(e,t){return this.storage.mod(e,t),this},m.prototype.modLayer=function(e,t){return this.painter.modLayer(e,t),this},m.prototype.addHoverShape=function(e){return this.storage.addHover(e),this},m.prototype.render=function(e){return this.painter.render(e),this._needsRefreshNextFrame=!1,this},m.prototype.refresh=function(e){return this.painter.refresh(e),this._needsRefreshNextFrame=!1,this},m.prototype.refreshNextFrame=function(){return this._needsRefreshNextFrame=!0,this},m.prototype.refreshHover=function(e){return this.painter.refreshHover(e),this},m.prototype.refreshShapes=function(e,t){return this.painter.refreshShapes(e,t),this},m.prototype.resize=function(){return this.painter.resize(),this},m.prototype.animate=function(e,t,a){if("string"==typeof e&&(e=this.storage.get(e)),e){var o;if(t){for(var r=t.split("."),s=e,l=0,h=r.length;h>l;l++)s&&(s=s[r[l]]);s&&(o=s)}else o=e;if(!o)return void n('Property "'+t+'" is not existed in element '+e.id);var d=this.animatingElements;return"undefined"==typeof e.__aniCount&&(e.__aniCount=0),0===e.__aniCount&&d.push(e),e.__aniCount++,this.animation.animate(o,{loop:a}).done(function(){if(e.__aniCount--,0===e.__aniCount){var t=i.indexOf(d,e);d.splice(t,1)}})}n("Element not existed")},m.prototype.clearAnimation=function(){this.animation.clear()},m.prototype.showLoading=function(e){return this.painter.showLoading(e),this},m.prototype.hideLoading=function(){return this.painter.hideLoading(),this},m.prototype.getWidth=function(){return this.painter.getWidth()},m.prototype.getHeight=function(){return this.painter.getHeight()},m.prototype.toDataURL=function(e,t,i){return this.painter.toDataURL(e,t,i)},m.prototype.shapeToImage=function(e,t,i){var n=a();return this.painter.shapeToImage(n,e,t,i)},m.prototype.on=function(e,t){return this.handler.on(e,t),this},m.prototype.un=function(e,t){return this.handler.un(e,t),this},m.prototype.trigger=function(e,t){return this.handler.trigger(e,t),this},m.prototype.clear=function(){return this.storage.delRoot(),this.painter.clear(),this},m.prototype.dispose=function(){this.animation.stop(),this.clear(),this.storage.dispose(),this.painter.dispose(),this.handler.dispose(),this.animation=this.animatingElements=this.storage=this.painter=this.handler=null,d.delInstance(this.id)},d}),define("zrender/config",[],function(){var e={EVENT:{RESIZE:"resize",CLICK:"click",DBLCLICK:"dblclick",MOUSEWHEEL:"mousewheel",MOUSEMOVE:"mousemove",MOUSEOVER:"mouseover",MOUSEOUT:"mouseout",MOUSEDOWN:"mousedown",MOUSEUP:"mouseup",GLOBALOUT:"globalout",DRAGSTART:"dragstart",DRAGEND:"dragend",DRAGENTER:"dragenter",DRAGOVER:"dragover",DRAGLEAVE:"dragleave",DROP:"drop",touchClickDelay:300},catchBrushException:!1,debugMode:0};return e}),define("echarts/chart/island",["require","../component/base","./base","zrender/shape/Circle","../config","../util/ecData","zrender/tool/util","zrender/tool/event","zrender/tool/color","../util/accMath","../chart"],function(e){function t(e,t,a,o,s){i.call(this,e,t,a,{},s),n.call(this),this._nameConnector,this._valueConnector,this._zrHeight=this.zr.getHeight(),this._zrWidth=this.zr.getWidth();var h=this;h.shapeHandler.onmousewheel=function(e){var t=e.target,i=e.event,n=l.getDelta(i);n=n>0?-1:1,t.style.r-=n,t.style.r=t.style.r<5?5:t.style.r;var a=r.get(t,"value"),o=a*h.option.island.calculateStep;a=o>1?Math.round(a-o*n):(a-o*n).toFixed(2)-0;var s=r.get(t,"name");t.style.text=s+":"+a,r.set(t,"value",a),r.set(t,"name",s),h.zr.modShape(t.id),h.zr.refresh(),l.stop(i)}}var i=e("../component/base"),n=e("./base"),a=e("zrender/shape/Circle"),o=e("../config"),r=e("../util/ecData"),s=e("zrender/tool/util"),l=e("zrender/tool/event");return t.prototype={type:o.CHART_TYPE_ISLAND,_combine:function(t,i){var n=e("zrender/tool/color"),a=e("../util/accMath"),o=a.accAdd(r.get(t,"value"),r.get(i,"value")),s=r.get(t,"name")+this._nameConnector+r.get(i,"name");t.style.text=s+this._valueConnector+o,r.set(t,"value",o),r.set(t,"name",s),t.style.r=this.option.island.r,t.style.color=n.mix(t.style.color,i.style.color)},refresh:function(e){e&&(e.island=this.reformOption(e.island),this.option=e,this._nameConnector=this.option.nameConnector,this._valueConnector=this.option.valueConnector)},getOption:function(){return this.option},resize:function(){var e=this.zr.getWidth(),t=this.zr.getHeight(),i=e/(this._zrWidth||e),n=t/(this._zrHeight||t);if(1!==i||1!==n){this._zrWidth=e,this._zrHeight=t;for(var a=0,o=this.shapeList.length;o>a;a++)this.zr.modShape(this.shapeList[a].id,{style:{x:Math.round(this.shapeList[a].style.x*i),y:Math.round(this.shapeList[a].style.y*n)}})}},add:function(e){var t=r.get(e,"name"),i=r.get(e,"value"),n=null!=r.get(e,"series")?r.get(e,"series").name:"",o=this.getFont(this.option.island.textStyle),s={zlevel:this._zlevelBase,style:{x:e.style.x,y:e.style.y,r:this.option.island.r,color:e.style.color||e.style.strokeColor,text:t+this._valueConnector+i,textFont:o},draggable:!0,hoverable:!0,onmousewheel:this.shapeHandler.onmousewheel,_type:"island"};"#fff"===s.style.color&&(s.style.color=e.style.strokeColor),this.setCalculable(s),s.dragEnableTime=0,r.pack(s,{name:n},-1,i,-1,t),s=new a(s),this.shapeList.push(s),this.zr.addShape(s)},del:function(e){this.zr.delShape(e.id);for(var t=[],i=0,n=this.shapeList.length;n>i;i++)this.shapeList[i].id!=e.id&&t.push(this.shapeList[i]);this.shapeList=t},ondrop:function(e,t){if(this.isDrop&&e.target){var i=e.target,n=e.dragged;this._combine(i,n),this.zr.modShape(i.id),t.dragIn=!0,this.isDrop=!1}},ondragend:function(e,t){var i=e.target;this.isDragend?t.dragIn&&(this.del(i),t.needRefresh=!0):t.dragIn||(i.style.x=l.getX(e.event),i.style.y=l.getY(e.event),this.add(i),t.needRefresh=!0),this.isDragend=!1}},s.inherits(t,n),s.inherits(t,i),e("../chart").define("island",t),t}),define("echarts/component/toolbox",["require","./base","zrender/shape/Line","zrender/shape/Image","zrender/shape/Rectangle","../util/shape/Icon","../config","zrender/tool/util","zrender/config","zrender/tool/event","./dataView","../component"],function(e){function t(e,t,n,a,o){i.call(this,e,t,n,a,o),this.dom=o.dom,this._magicType={},this._magicMap={},this._isSilence=!1,this._iconList,this._iconShapeMap={},this._featureTitle={},this._featureIcon={},this._featureColor={},this._featureOption={},this._enableColor="red",this._disableColor="#ccc",this._markShapeList=[];var r=this;r._onMark=function(e){r.__onMark(e)},r._onMarkUndo=function(e){r.__onMarkUndo(e)},r._onMarkClear=function(e){r.__onMarkClear(e)},r._onDataZoom=function(e){r.__onDataZoom(e)},r._onDataZoomReset=function(e){r.__onDataZoomReset(e)},r._onDataView=function(e){r.__onDataView(e)},r._onRestore=function(e){r.__onRestore(e)},r._onSaveAsImage=function(e){r.__onSaveAsImage(e)},r._onMagicType=function(e){r.__onMagicType(e)},r._onCustomHandler=function(e){r.__onCustomHandler(e)},r._onmousemove=function(e){return r.__onmousemove(e)},r._onmousedown=function(e){return r.__onmousedown(e)},r._onmouseup=function(e){return r.__onmouseup(e)},r._onclick=function(e){return r.__onclick(e)}}var i=e("./base"),n=e("zrender/shape/Line"),a=e("zrender/shape/Image"),o=e("zrender/shape/Rectangle"),r=e("../util/shape/Icon"),s=e("../config"),l=e("zrender/tool/util"),h=e("zrender/config"),d=e("zrender/tool/event"),m="stack",c="tiled";return t.prototype={type:s.COMPONENT_TYPE_TOOLBOX,_buildShape:function(){this._iconList=[];var e=this.option.toolbox;this._enableColor=e.effectiveColor,this._disableColor=e.disableColor;var t=e.feature,i=[];for(var n in t)if(t[n].show)switch(n){case"mark":i.push({key:n,name:"mark"}),i.push({key:n,name:"markUndo"}),i.push({key:n,name:"markClear"});break;case"magicType":for(var a=0,o=t[n].type.length;o>a;a++)t[n].title[t[n].type[a]+"Chart"]=t[n].title[t[n].type[a]],t[n].option&&(t[n].option[t[n].type[a]+"Chart"]=t[n].option[t[n].type[a]]),i.push({key:n,name:t[n].type[a]+"Chart"});break;case"dataZoom":i.push({key:n,name:"dataZoom"}),i.push({key:n,name:"dataZoomReset"});break;case"saveAsImage":this.canvasSupported&&i.push({key:n,name:"saveAsImage"});break;default:i.push({key:n,name:n})}if(i.length>0){for(var r,n,a=0,o=i.length;o>a;a++)r=i[a].name,n=i[a].key,this._iconList.push(r),this._featureTitle[r]=t[n].title[r]||t[n].title,t[n].icon&&(this._featureIcon[r]=t[n].icon[r]||t[n].icon),t[n].color&&(this._featureColor[r]=t[n].color[r]||t[n].color),t[n].option&&(this._featureOption[r]=t[n].option[r]||t[n].option);this._itemGroupLocation=this._getItemGroupLocation(),this._buildBackground(),this._buildItem();for(var a=0,o=this.shapeList.length;o>a;a++)this.zr.addShape(this.shapeList[a]);this._iconShapeMap.mark&&(this._iconDisable(this._iconShapeMap.markUndo),this._iconDisable(this._iconShapeMap.markClear)),this._iconShapeMap.dataZoomReset&&0===this._zoomQueue.length&&this._iconDisable(this._iconShapeMap.dataZoomReset)}},_buildItem:function(){var t,i,n,o,s=this.option.toolbox,l=this._iconList.length,h=this._itemGroupLocation.x,d=this._itemGroupLocation.y,m=s.itemSize,c=s.itemGap,p=s.color instanceof Array?s.color:[s.color],u=this.getFont(s.textStyle);"horizontal"===s.orient?(i=this._itemGroupLocation.y/this.zr.getHeight()<.5?"bottom":"top",n=this._itemGroupLocation.x/this.zr.getWidth()<.5?"left":"right",o=this._itemGroupLocation.y/this.zr.getHeight()<.5?"top":"bottom"):i=this._itemGroupLocation.x/this.zr.getWidth()<.5?"right":"left",this._iconShapeMap={};for(var V=this,U=0;l>U;U++){switch(t={type:"icon",zlevel:this._zlevelBase,style:{x:h,y:d,width:m,height:m,iconType:this._iconList[U],lineWidth:1,strokeColor:this._featureColor[this._iconList[U]]||p[U%p.length],brushType:"stroke"},highlightStyle:{lineWidth:1,text:s.showTitle?this._featureTitle[this._iconList[U]]:void 0,textFont:u,textPosition:i,strokeColor:this._featureColor[this._iconList[U]]||p[U%p.length]},hoverable:!0,clickable:!0},this._featureIcon[this._iconList[U]]&&(t.style.image=this._featureIcon[this._iconList[U]].replace(new RegExp("^image:\\/\\/"),""),t.style.opacity=.8,t.highlightStyle.opacity=1,t.type="image"),"horizontal"===s.orient&&(0===U&&"left"===n&&(t.highlightStyle.textPosition="specific",t.highlightStyle.textAlign=n,t.highlightStyle.textBaseline=o,t.highlightStyle.textX=h,t.highlightStyle.textY="top"===o?d+m+10:d-10),U===l-1&&"right"===n&&(t.highlightStyle.textPosition="specific",t.highlightStyle.textAlign=n,t.highlightStyle.textBaseline=o,t.highlightStyle.textX=h+m,t.highlightStyle.textY="top"===o?d+m+10:d-10)),this._iconList[U]){case"mark":t.onclick=V._onMark;break;case"markUndo":t.onclick=V._onMarkUndo;break;case"markClear":t.onclick=V._onMarkClear;break;case"dataZoom":t.onclick=V._onDataZoom;break;case"dataZoomReset":t.onclick=V._onDataZoomReset;break;case"dataView":if(!this._dataView){var y=e("./dataView");this._dataView=new y(this.ecTheme,this.messageCenter,this.zr,this.option,this.myChart)}t.onclick=V._onDataView;break;case"restore":t.onclick=V._onRestore;break;case"saveAsImage":t.onclick=V._onSaveAsImage;break;default:this._iconList[U].match("Chart")?(t._name=this._iconList[U].replace("Chart",""),t.onclick=V._onMagicType):t.onclick=V._onCustomHandler}"icon"===t.type?t=new r(t):"image"===t.type&&(t=new a(t)),this.shapeList.push(t),this._iconShapeMap[this._iconList[U]]=t,"horizontal"===s.orient?h+=m+c:d+=m+c}},_buildBackground:function(){var e=this.option.toolbox,t=this.reformCssArray(this.option.toolbox.padding);this.shapeList.push(new o({zlevel:this._zlevelBase,hoverable:!1,style:{x:this._itemGroupLocation.x-t[3],y:this._itemGroupLocation.y-t[0],width:this._itemGroupLocation.width+t[3]+t[1],height:this._itemGroupLocation.height+t[0]+t[2],brushType:0===e.borderWidth?"fill":"both",color:e.backgroundColor,strokeColor:e.borderColor,lineWidth:e.borderWidth}}))},_getItemGroupLocation:function(){var e=this.option.toolbox,t=this.reformCssArray(this.option.toolbox.padding),i=this._iconList.length,n=e.itemGap,a=e.itemSize,o=0,r=0;"horizontal"===e.orient?(o=(a+n)*i-n,r=a):(r=(a+n)*i-n,o=a);var s,l=this.zr.getWidth();switch(e.x){case"center":s=Math.floor((l-o)/2);break;case"left":s=t[3]+e.borderWidth;break;case"right":s=l-o-t[1]-e.borderWidth;break;default:s=e.x-0,s=isNaN(s)?0:s}var h,d=this.zr.getHeight();switch(e.y){case"top":h=t[0]+e.borderWidth;break;case"bottom":h=d-r-t[2]-e.borderWidth;break;case"center":h=Math.floor((d-r)/2);break;default:h=e.y-0,h=isNaN(h)?0:h}return{x:s,y:h,width:o,height:r}},__onmousemove:function(e){this._marking&&(this._markShape.style.xEnd=d.getX(e.event),this._markShape.style.yEnd=d.getY(e.event),this.zr.addHoverShape(this._markShape)),this._zooming&&(this._zoomShape.style.width=d.getX(e.event)-this._zoomShape.style.x,this._zoomShape.style.height=d.getY(e.event)-this._zoomShape.style.y,this.zr.addHoverShape(this._zoomShape),this.dom.style.cursor="crosshair"),this._zoomStart&&"pointer"!=this.dom.style.cursor&&"move"!=this.dom.style.cursor&&(this.dom.style.cursor="crosshair")},__onmousedown:function(e){if(!e.target){this._zooming=!0;var t=d.getX(e.event),i=d.getY(e.event),n=this.option.dataZoom||{};return this._zoomShape=new o({zlevel:this._zlevelBase,style:{x:t,y:i,width:1,height:1,brushType:"both"},highlightStyle:{lineWidth:2,color:n.fillerColor||s.dataZoom.fillerColor,strokeColor:n.handleColor||s.dataZoom.handleColor,brushType:"both"}}),this.zr.addHoverShape(this._zoomShape),!0}},__onmouseup:function(){if(!this._zoomShape||Math.abs(this._zoomShape.style.width)<10||Math.abs(this._zoomShape.style.height)<10)return this._zooming=!1,!0;if(this._zooming&&this.component.dataZoom){this._zooming=!1;var e=this.component.dataZoom.rectZoom(this._zoomShape.style);e&&(this._zoomQueue.push({start:e.start,end:e.end,start2:e.start2,end2:e.end2}),this._iconEnable(this._iconShapeMap.dataZoomReset),this.zr.refresh())}return!0},__onclick:function(e){if(!e.target)if(this._marking)this._marking=!1,this._markShapeList.push(this._markShape),this._iconEnable(this._iconShapeMap.markUndo),this._iconEnable(this._iconShapeMap.markClear),this.zr.addShape(this._markShape),this.zr.refresh();else if(this._markStart){this._marking=!0;var t=d.getX(e.event),i=d.getY(e.event);this._markShape=new n({zlevel:this._zlevelBase,style:{xStart:t,yStart:i,xEnd:t,yEnd:i,lineWidth:this.query(this.option,"toolbox.feature.mark.lineStyle.width"),strokeColor:this.query(this.option,"toolbox.feature.mark.lineStyle.color"),lineType:this.query(this.option,"toolbox.feature.mark.lineStyle.type")}}),this.zr.addHoverShape(this._markShape)}},__onMark:function(e){var t=e.target;if(this._marking||this._markStart)this._resetMark(),this.zr.refresh();else{this._resetZoom(),this.zr.modShape(t.id,{style:{strokeColor:this._enableColor}}),this.zr.refresh(),this._markStart=!0;var i=this;setTimeout(function(){i.zr&&i.zr.on(h.EVENT.CLICK,i._onclick)&&i.zr.on(h.EVENT.MOUSEMOVE,i._onmousemove)},10)}return!0},__onMarkUndo:function(){if(this._marking)this._marking=!1;else{var e=this._markShapeList.length;if(e>=1){var t=this._markShapeList[e-1];this.zr.delShape(t.id),this.zr.refresh(),this._markShapeList.pop(),1===e&&(this._iconDisable(this._iconShapeMap.markUndo),this._iconDisable(this._iconShapeMap.markClear))}}return!0},__onMarkClear:function(){this._marking&&(this._marking=!1);var e=this._markShapeList.length;if(e>0){for(;e--;)this.zr.delShape(this._markShapeList.pop().id);this._iconDisable(this._iconShapeMap.markUndo),this._iconDisable(this._iconShapeMap.markClear),this.zr.refresh()}return!0},__onDataZoom:function(e){var t=e.target;if(this._zooming||this._zoomStart)this._resetZoom(),this.zr.refresh(),this.dom.style.cursor="default";else{this._resetMark(),this.zr.modShape(t.id,{style:{strokeColor:this._enableColor}}),this.zr.refresh(),this._zoomStart=!0;var i=this;setTimeout(function(){i.zr&&i.zr.on(h.EVENT.MOUSEDOWN,i._onmousedown)&&i.zr.on(h.EVENT.MOUSEUP,i._onmouseup)&&i.zr.on(h.EVENT.MOUSEMOVE,i._onmousemove)},10),this.dom.style.cursor="crosshair"}return!0},__onDataZoomReset:function(){return this._zooming&&(this._zooming=!1),this._zoomQueue.pop(),this._zoomQueue.length>0?this.component.dataZoom.absoluteZoom(this._zoomQueue[this._zoomQueue.length-1]):(this.component.dataZoom.rectZoom(),this._iconDisable(this._iconShapeMap.dataZoomReset),this.zr.refresh()),!0},_resetMark:function(){this._marking=!1,this._markStart&&(this._markStart=!1,this._iconShapeMap.mark&&this.zr.modShape(this._iconShapeMap.mark.id,{style:{strokeColor:this._iconShapeMap.mark.highlightStyle.strokeColor}}),this.zr.un(h.EVENT.CLICK,this._onclick),this.zr.un(h.EVENT.MOUSEMOVE,this._onmousemove))},_resetZoom:function(){this._zooming=!1,this._zoomStart&&(this._zoomStart=!1,this._iconShapeMap.dataZoom&&this.zr.modShape(this._iconShapeMap.dataZoom.id,{style:{strokeColor:this._iconShapeMap.dataZoom.highlightStyle.strokeColor}}),this.zr.un(h.EVENT.MOUSEDOWN,this._onmousedown),this.zr.un(h.EVENT.MOUSEUP,this._onmouseup),this.zr.un(h.EVENT.MOUSEMOVE,this._onmousemove))},_iconDisable:function(e){"image"!=e.type?this.zr.modShape(e.id,{hoverable:!1,clickable:!1,style:{strokeColor:this._disableColor}}):this.zr.modShape(e.id,{hoverable:!1,clickable:!1,style:{opacity:.3}})},_iconEnable:function(e){"image"!=e.type?this.zr.modShape(e.id,{hoverable:!0,clickable:!0,style:{strokeColor:e.highlightStyle.strokeColor}}):this.zr.modShape(e.id,{hoverable:!0,clickable:!0,style:{opacity:.8}})},__onDataView:function(){return this._dataView.show(this.option),!0},__onRestore:function(){return this._resetMark(),this._resetZoom(),this.messageCenter.dispatch(s.EVENT.RESTORE,null,null,this.myChart),!0},__onSaveAsImage:function(){var e=this.option.toolbox.feature.saveAsImage,t=e.type||"png";"png"!=t&&"jpeg"!=t&&(t="png");var i;i=this.myChart.isConnected()?this.myChart.getConnectedDataURL(t):this.zr.toDataURL("image/"+t,this.option.backgroundColor&&"rgba(0,0,0,0)"===this.option.backgroundColor.replace(" ","")?"#fff":this.option.backgroundColor);var n=document.createElement("div");n.id="__echarts_download_wrap__",n.style.cssText="position:fixed;z-index:99999;display:block;top:0;left:0;background-color:rgba(33,33,33,0.5);text-align:center;width:100%;height:100%;line-height:"+document.documentElement.clientHeight+"px;";var a=document.createElement("a");a.href=i,a.setAttribute("download",(e.name?e.name:this.option.title&&(this.option.title.text||this.option.title.subtext)?this.option.title.text||this.option.title.subtext:"ECharts")+"."+t),a.innerHTML='Save as":e.lang?e.lang[0]:"Save")+'"/>',n.appendChild(a),document.body.appendChild(n),a=null,n=null,setTimeout(function(){var e=document.getElementById("__echarts_download_wrap__");e&&(e.onclick=function(){var e=document.getElementById("__echarts_download_wrap__");e.onclick=null,e.innerHTML="",document.body.removeChild(e),e=null},e=null)},500)},__onMagicType:function(e){this._resetMark();var t=e.target._name;return this._magicType[t]||(this._magicType[t]=!0,t===s.CHART_TYPE_LINE?this._magicType[s.CHART_TYPE_BAR]=!1:t===s.CHART_TYPE_BAR&&(this._magicType[s.CHART_TYPE_LINE]=!1),t===s.CHART_TYPE_PIE?this._magicType[s.CHART_TYPE_FUNNEL]=!1:t===s.CHART_TYPE_FUNNEL&&(this._magicType[s.CHART_TYPE_PIE]=!1),t===s.CHART_TYPE_FORCE?this._magicType[s.CHART_TYPE_CHORD]=!1:t===s.CHART_TYPE_CHORD&&(this._magicType[s.CHART_TYPE_FORCE]=!1),t===m?this._magicType[c]=!1:t===c&&(this._magicType[m]=!1),this.messageCenter.dispatch(s.EVENT.MAGIC_TYPE_CHANGED,e.event,{magicType:this._magicType},this.myChart)),!0},setMagicType:function(e){this._resetMark(),this._magicType=e,!this._isSilence&&this.messageCenter.dispatch(s.EVENT.MAGIC_TYPE_CHANGED,null,{magicType:this._magicType},this.myChart)},__onCustomHandler:function(e){var t=e.target.style.iconType,i=this.option.toolbox.feature[t].onclick;"function"==typeof i&&i.call(this,this.option)},reset:function(e,t){if(t&&this.clear(),this.query(e,"toolbox.show")&&this.query(e,"toolbox.feature.magicType.show")){var i=e.toolbox.feature.magicType.type,n=i.length;for(this._magicMap={};n--;)this._magicMap[i[n]]=!0;n=e.series.length;for(var a,o;n--;)a=e.series[n].type,this._magicMap[a]&&(o=e.xAxis instanceof Array?e.xAxis[e.series[n].xAxisIndex||0]:e.xAxis,o&&"category"===(o.type||"category")&&(o.__boundaryGap=null!=o.boundaryGap?o.boundaryGap:!0),o=e.yAxis instanceof Array?e.yAxis[e.series[n].yAxisIndex||0]:e.yAxis,o&&"category"===o.type&&(o.__boundaryGap=null!=o.boundaryGap?o.boundaryGap:!0),e.series[n].__type=a,e.series[n].__itemStyle=l.clone(e.series[n].itemStyle||{})),(this._magicMap[m]||this._magicMap[c])&&(e.series[n].__stack=e.series[n].stack)}this._magicType=t?{}:this._magicType||{};for(var r in this._magicType)if(this._magicType[r]){this.option=e,this.getMagicOption();break}var s=e.dataZoom;if(s&&s.show){var h=null!=s.start&&s.start>=0&&s.start<=100?s.start:0,d=null!=s.end&&s.end>=0&&s.end<=100?s.end:100;h>d&&(h+=d,d=h-d,h-=d),this._zoomQueue=[{start:h,end:d,start2:0,end2:100}]}else this._zoomQueue=[]},getMagicOption:function(){var e,t; +if(this._magicType[s.CHART_TYPE_LINE]||this._magicType[s.CHART_TYPE_BAR]){for(var i=this._magicType[s.CHART_TYPE_LINE]?!1:!0,n=0,a=this.option.series.length;a>n;n++)t=this.option.series[n].type,(t==s.CHART_TYPE_LINE||t==s.CHART_TYPE_BAR)&&(e=this.option.xAxis instanceof Array?this.option.xAxis[this.option.series[n].xAxisIndex||0]:this.option.xAxis,e&&"category"===(e.type||"category")&&(e.boundaryGap=i?!0:e.__boundaryGap),e=this.option.yAxis instanceof Array?this.option.yAxis[this.option.series[n].yAxisIndex||0]:this.option.yAxis,e&&"category"===e.type&&(e.boundaryGap=i?!0:e.__boundaryGap));this._defaultMagic(s.CHART_TYPE_LINE,s.CHART_TYPE_BAR)}if(this._defaultMagic(s.CHART_TYPE_CHORD,s.CHART_TYPE_FORCE),this._defaultMagic(s.CHART_TYPE_PIE,s.CHART_TYPE_FUNNEL),this._magicType[m]||this._magicType[c])for(var n=0,a=this.option.series.length;a>n;n++)this._magicType[m]?(this.option.series[n].stack="_ECHARTS_STACK_KENER_2014_",t=m):this._magicType[c]&&(this.option.series[n].stack=null,t=c),this._featureOption[t+"Chart"]&&l.merge(this.option.series[n],this._featureOption[t+"Chart"]||{},!0);return this.option},_defaultMagic:function(e,t){if(this._magicType[e]||this._magicType[t])for(var i=0,n=this.option.series.length;n>i;i++){var a=this.option.series[i].type;(a==e||a==t)&&(this.option.series[i].type=this._magicType[e]?e:t,this.option.series[i].itemStyle=l.clone(this.option.series[i].__itemStyle),a=this.option.series[i].type,this._featureOption[a+"Chart"]&&l.merge(this.option.series[i],this._featureOption[a+"Chart"]||{},!0))}},silence:function(e){this._isSilence=e},resize:function(){this._resetMark(),this.clear(),this.option&&this.option.toolbox&&this.option.toolbox.show&&this._buildShape(),this._dataView&&this._dataView.resize()},hideDataView:function(){this._dataView&&this._dataView.hide()},clear:function(e){this.zr&&(this.zr.delShape(this.shapeList),this.shapeList=[],e||(this.zr.delShape(this._markShapeList),this._markShapeList=[]))},onbeforDispose:function(){this._dataView&&(this._dataView.dispose(),this._dataView=null),this._markShapeList=null},refresh:function(e){e&&(this._resetMark(),this._resetZoom(),e.toolbox=this.reformOption(e.toolbox),this.option=e,this.clear(!0),e.toolbox.show&&this._buildShape(),this.hideDataView())}},l.inherits(t,i),e("../component").define("toolbox",t),t}),define("echarts/component",[],function(){var e={},t={};return e.define=function(i,n){return t[i]=n,e},e.get=function(e){return t[e]},e}),define("echarts/component/title",["require","./base","zrender/shape/Text","zrender/shape/Rectangle","../config","zrender/tool/util","zrender/tool/area","zrender/tool/color","../component"],function(e){function t(e,t,n,a,o){i.call(this,e,t,n,a,o),this.refresh(a)}var i=e("./base"),n=e("zrender/shape/Text"),a=e("zrender/shape/Rectangle"),o=e("../config"),r=e("zrender/tool/util"),s=e("zrender/tool/area"),l=e("zrender/tool/color");return t.prototype={type:o.COMPONENT_TYPE_TITLE,_buildShape:function(){this._itemGroupLocation=this._getItemGroupLocation(),this._buildBackground(),this._buildItem();for(var e=0,t=this.shapeList.length;t>e;e++)this.zr.addShape(this.shapeList[e])},_buildItem:function(){var e=this.titleOption.text,t=this.titleOption.link,i=this.titleOption.target,a=this.titleOption.subtext,o=this.titleOption.sublink,r=this.titleOption.subtarget,s=this.getFont(this.titleOption.textStyle),h=this.getFont(this.titleOption.subtextStyle),d=this._itemGroupLocation.x,m=this._itemGroupLocation.y,c=this._itemGroupLocation.width,p=this._itemGroupLocation.height,u={zlevel:this._zlevelBase,style:{y:m,color:this.titleOption.textStyle.color,text:e,textFont:s,textBaseline:"top"},highlightStyle:{color:l.lift(this.titleOption.textStyle.color,1),brushType:"fill"},hoverable:!1};t&&(u.hoverable=!0,u.clickable=!0,u.onclick=function(){i&&"self"==i?window.location=t:window.open(t)});var V={zlevel:this._zlevelBase,style:{y:m+p,color:this.titleOption.subtextStyle.color,text:a,textFont:h,textBaseline:"bottom"},highlightStyle:{color:l.lift(this.titleOption.subtextStyle.color,1),brushType:"fill"},hoverable:!1};switch(o&&(V.hoverable=!0,V.clickable=!0,V.onclick=function(){r&&"self"==r?window.location=o:window.open(o)}),this.titleOption.x){case"center":u.style.x=V.style.x=d+c/2,u.style.textAlign=V.style.textAlign="center";break;case"left":u.style.x=V.style.x=d,u.style.textAlign=V.style.textAlign="left";break;case"right":u.style.x=V.style.x=d+c,u.style.textAlign=V.style.textAlign="right";break;default:d=this.titleOption.x-0,d=isNaN(d)?0:d,u.style.x=V.style.x=d}this.titleOption.textAlign&&(u.style.textAlign=V.style.textAlign=this.titleOption.textAlign),this.shapeList.push(new n(u)),""!==a&&this.shapeList.push(new n(V))},_buildBackground:function(){var e=this.reformCssArray(this.titleOption.padding);this.shapeList.push(new a({zlevel:this._zlevelBase,hoverable:!1,style:{x:this._itemGroupLocation.x-e[3],y:this._itemGroupLocation.y-e[0],width:this._itemGroupLocation.width+e[3]+e[1],height:this._itemGroupLocation.height+e[0]+e[2],brushType:0===this.titleOption.borderWidth?"fill":"both",color:this.titleOption.backgroundColor,strokeColor:this.titleOption.borderColor,lineWidth:this.titleOption.borderWidth}}))},_getItemGroupLocation:function(){var e,t=this.reformCssArray(this.titleOption.padding),i=this.titleOption.text,n=this.titleOption.subtext,a=this.getFont(this.titleOption.textStyle),o=this.getFont(this.titleOption.subtextStyle),r=Math.max(s.getTextWidth(i,a),s.getTextWidth(n,o)),l=s.getTextHeight(i,a)+(""===n?0:this.titleOption.itemGap+s.getTextHeight(n,o)),h=this.zr.getWidth();switch(this.titleOption.x){case"center":e=Math.floor((h-r)/2);break;case"left":e=t[3]+this.titleOption.borderWidth;break;case"right":e=h-r-t[1]-this.titleOption.borderWidth;break;default:e=this.titleOption.x-0,e=isNaN(e)?0:e}var d,m=this.zr.getHeight();switch(this.titleOption.y){case"top":d=t[0]+this.titleOption.borderWidth;break;case"bottom":d=m-l-t[2]-this.titleOption.borderWidth;break;case"center":d=Math.floor((m-l)/2);break;default:d=this.titleOption.y-0,d=isNaN(d)?0:d}return{x:e,y:d,width:r,height:l}},refresh:function(e){e&&(this.option=e,this.option.title=this.reformOption(this.option.title),this.titleOption=this.option.title,this.titleOption.textStyle=r.merge(this.titleOption.textStyle,this.ecTheme.textStyle),this.titleOption.subtextStyle=r.merge(this.titleOption.subtextStyle,this.ecTheme.textStyle)),this.clear(),this._buildShape()}},r.inherits(t,i),e("../component").define("title",t),t}),define("echarts/component/tooltip",["require","./base","../util/shape/Cross","zrender/shape/Line","zrender/shape/Rectangle","../config","../util/ecData","zrender/config","zrender/tool/event","zrender/tool/area","zrender/tool/color","zrender/tool/util","zrender/shape/Base","../component"],function(e){function t(e,t,o,r,s){i.call(this,e,t,o,r,s),this.dom=s.dom;var l=this;l._onmousemove=function(e){return l.__onmousemove(e)},l._onglobalout=function(e){return l.__onglobalout(e)},this.zr.on(h.EVENT.MOUSEMOVE,l._onmousemove),this.zr.on(h.EVENT.GLOBALOUT,l._onglobalout),l._hide=function(e){return l.__hide(e)},l._tryShow=function(e){return l.__tryShow(e)},l._refixed=function(e){return l.__refixed(e)},l._setContent=function(e,t){return l.__setContent(e,t)},this._tDom=this._tDom||document.createElement("div"),this._tDom.onselectstart=function(){return!1},this._tDom.onmouseover=function(){l._mousein=!0},this._tDom.onmouseout=function(){l._mousein=!1},this._tDom.style.position="absolute",this.hasAppend=!1,this._axisLineShape&&this.zr.delShape(this._axisLineShape.id),this._axisLineShape=new a({zlevel:this._zlevelBase,invisible:!0,hoverable:!1}),this.shapeList.push(this._axisLineShape),this.zr.addShape(this._axisLineShape),this._axisShadowShape&&this.zr.delShape(this._axisShadowShape.id),this._axisShadowShape=new a({zlevel:1,invisible:!0,hoverable:!1}),this.shapeList.push(this._axisShadowShape),this.zr.addShape(this._axisShadowShape),this._axisCrossShape&&this.zr.delShape(this._axisCrossShape.id),this._axisCrossShape=new n({zlevel:this._zlevelBase,invisible:!0,hoverable:!1}),this.shapeList.push(this._axisCrossShape),this.zr.addShape(this._axisCrossShape),this.showing=!1,this.refresh(r)}var i=e("./base"),n=e("../util/shape/Cross"),a=e("zrender/shape/Line"),o=e("zrender/shape/Rectangle"),r=new o({}),s=e("../config"),l=e("../util/ecData"),h=e("zrender/config"),d=e("zrender/tool/event"),m=e("zrender/tool/area"),c=e("zrender/tool/color"),p=e("zrender/tool/util"),u=e("zrender/shape/Base");return t.prototype={type:s.COMPONENT_TYPE_TOOLTIP,_gCssText:"position:absolute;display:block;border-style:solid;white-space:nowrap;",_style:function(e){if(!e)return"";var t=[];if(e.transitionDuration){var i="left "+e.transitionDuration+"s,top "+e.transitionDuration+"s";t.push("transition:"+i),t.push("-moz-transition:"+i),t.push("-webkit-transition:"+i),t.push("-o-transition:"+i)}e.backgroundColor&&(t.push("background-Color:"+c.toHex(e.backgroundColor)),t.push("filter:alpha(opacity=70)"),t.push("background-Color:"+e.backgroundColor)),null!=e.borderWidth&&t.push("border-width:"+e.borderWidth+"px"),null!=e.borderColor&&t.push("border-color:"+e.borderColor),null!=e.borderRadius&&(t.push("border-radius:"+e.borderRadius+"px"),t.push("-moz-border-radius:"+e.borderRadius+"px"),t.push("-webkit-border-radius:"+e.borderRadius+"px"),t.push("-o-border-radius:"+e.borderRadius+"px"));var n=e.textStyle;n&&(n.color&&t.push("color:"+n.color),n.decoration&&t.push("text-decoration:"+n.decoration),n.align&&t.push("text-align:"+n.align),n.fontFamily&&t.push("font-family:"+n.fontFamily),n.fontSize&&t.push("font-size:"+n.fontSize+"px"),n.fontSize&&t.push("line-height:"+Math.round(3*n.fontSize/2)+"px"),n.fontStyle&&t.push("font-style:"+n.fontStyle),n.fontWeight&&t.push("font-weight:"+n.fontWeight));var a=e.padding;return null!=a&&(a=this.reformCssArray(a),t.push("padding:"+a[0]+"px "+a[1]+"px "+a[2]+"px "+a[3]+"px")),t=t.join(";")+";"},__hide:function(){this._lastDataIndex=-1,this._lastSeriesIndex=-1,this._lastItemTriggerId=-1,this._tDom&&(this._tDom.style.display="none");var e=!1;this._axisLineShape.invisible||(this._axisLineShape.invisible=!0,this.zr.modShape(this._axisLineShape.id),e=!0),this._axisShadowShape.invisible||(this._axisShadowShape.invisible=!0,this.zr.modShape(this._axisShadowShape.id),e=!0),this._axisCrossShape.invisible||(this._axisCrossShape.invisible=!0,this.zr.modShape(this._axisCrossShape.id),e=!0),this._lastTipShape&&this._lastTipShape.tipShape.length>0&&(this.zr.delShape(this._lastTipShape.tipShape),this._lastTipShape=!1,this.shapeList.length=2),e&&this.zr.refresh(),this.showing=!1},_show:function(e,t,i,n){var a=this._tDom.offsetHeight,o=this._tDom.offsetWidth;e&&("function"==typeof e&&(e=e([t,i])),e instanceof Array&&(t=e[0],i=e[1])),t+o>this._zrWidth&&(t-=o+40),i+a>this._zrHeight&&(i-=a-20),20>i&&(i=0),this._tDom.style.cssText=this._gCssText+this._defaultCssText+(n?n:"")+"left:"+t+"px;top:"+i+"px;",(10>a||10>o)&&setTimeout(this._refixed,20),this.showing=!0},__refixed:function(){if(this._tDom){var e="",t=this._tDom.offsetHeight,i=this._tDom.offsetWidth;this._tDom.offsetLeft+i>this._zrWidth&&(e+="left:"+(this._zrWidth-i-20)+"px;"),this._tDom.offsetTop+t>this._zrHeight&&(e+="top:"+(this._zrHeight-t-10)+"px;"),""!==e&&(this._tDom.style.cssText+=e)}},__tryShow:function(){var e,t;if(this._curTarget){if("island"===this._curTarget._type&&this.option.tooltip.show)return void this._showItemTrigger();var i=l.get(this._curTarget,"series"),n=l.get(this._curTarget,"data");e=this.deepQuery([n,i,this.option],"tooltip.show"),null!=i&&null!=n&&e?(t=this.deepQuery([n,i,this.option],"tooltip.trigger"),"axis"===t?this._showAxisTrigger(i.xAxisIndex,i.yAxisIndex,l.get(this._curTarget,"dataIndex")):this._showItemTrigger()):(clearTimeout(this._hidingTicket),clearTimeout(this._showingTicket),this._hidingTicket=setTimeout(this._hide,this._hideDelay))}else this._findPolarTrigger()||this._findAxisTrigger()},_findAxisTrigger:function(){if(!this.component.xAxis||!this.component.yAxis)return void(this._hidingTicket=setTimeout(this._hide,this._hideDelay));for(var e,t,i=this.option.series,n=0,a=i.length;a>n;n++)if("axis"===this.deepQuery([i[n],this.option],"tooltip.trigger"))return e=i[n].xAxisIndex||0,t=i[n].yAxisIndex||0,this.component.xAxis.getAxis(e)&&this.component.xAxis.getAxis(e).type===s.COMPONENT_TYPE_AXIS_CATEGORY?void this._showAxisTrigger(e,t,this._getNearestDataIndex("x",this.component.xAxis.getAxis(e))):this.component.yAxis.getAxis(t)&&this.component.yAxis.getAxis(t).type===s.COMPONENT_TYPE_AXIS_CATEGORY?void this._showAxisTrigger(e,t,this._getNearestDataIndex("y",this.component.yAxis.getAxis(t))):void this._showAxisTrigger(e,t,-1);"cross"===this.option.tooltip.axisPointer.type&&this._showAxisTrigger(-1,-1,-1)},_findPolarTrigger:function(){if(!this.component.polar)return!1;var e,t=d.getX(this._event),i=d.getY(this._event),n=this.component.polar.getNearestIndex([t,i]);return n?(e=n.valueIndex,n=n.polarIndex):n=-1,-1!=n?this._showPolarTrigger(n,e):!1},_getNearestDataIndex:function(e,t){var i=-1,n=d.getX(this._event),a=d.getY(this._event);if("x"===e){for(var o,r,s=this.component.grid.getXend(),l=t.getCoordByIndex(i);s>l&&(r=l,n>=l);)o=l,l=t.getCoordByIndex(++i);return 0>=i?i=0:r-n>=n-o?i-=1:null==t.getNameByIndex(i)&&(i-=1),i}for(var h,m,c=this.component.grid.getY(),l=t.getCoordByIndex(i);l>c&&(h=l,l>=a);)m=l,l=t.getCoordByIndex(++i);return 0>=i?i=0:a-h>=m-a?i-=1:null==t.getNameByIndex(i)&&(i-=1),i},_showAxisTrigger:function(e,t,i){if(!this._event.connectTrigger&&this.messageCenter.dispatch(s.EVENT.TOOLTIP_IN_GRID,this._event,null,this.myChart),null==this.component.xAxis||null==this.component.yAxis||null==e||null==t)return clearTimeout(this._hidingTicket),clearTimeout(this._showingTicket),void(this._hidingTicket=setTimeout(this._hide,this._hideDelay));var n,a,o,r,l=this.option.series,h=[],m=[],c="";if("axis"===this.option.tooltip.trigger){if(!this.option.tooltip.show)return;a=this.option.tooltip.formatter,o=this.option.tooltip.position}var p,u,V=-1!=e&&this.component.xAxis.getAxis(e).type===s.COMPONENT_TYPE_AXIS_CATEGORY?"xAxis":-1!=t&&this.component.yAxis.getAxis(t).type===s.COMPONENT_TYPE_AXIS_CATEGORY?"yAxis":!1;if(V){var U="xAxis"==V?e:t;n=this.component[V].getAxis(U);for(var y=0,g=l.length;g>y;y++)this._isSelected(l[y].name)&&l[y][V+"Index"]===U&&"axis"===this.deepQuery([l[y],this.option],"tooltip.trigger")&&(r=this.query(l[y],"tooltip.showContent")||r,a=this.query(l[y],"tooltip.formatter")||a,o=this.query(l[y],"tooltip.position")||o,c+=this._style(this.query(l[y],"tooltip")),null!=l[y].stack&&"xAxis"==V?(h.unshift(l[y]),m.unshift(y)):(h.push(l[y]),m.push(y)));this.messageCenter.dispatch(s.EVENT.TOOLTIP_HOVER,this._event,{seriesIndex:m,dataIndex:i},this.myChart);var f;"xAxis"==V?(p=this.subPixelOptimize(n.getCoordByIndex(i),this._axisLineWidth),u=d.getY(this._event),f=[p,this.component.grid.getY(),p,this.component.grid.getYend()]):(p=d.getX(this._event),u=this.subPixelOptimize(n.getCoordByIndex(i),this._axisLineWidth),f=[this.component.grid.getX(),u,this.component.grid.getXend(),u]),this._styleAxisPointer(h,f[0],f[1],f[2],f[3],n.getGap(),p,u)}else p=d.getX(this._event),u=d.getY(this._event),this._styleAxisPointer(l,this.component.grid.getX(),u,this.component.grid.getXend(),u,0,p,u),i>=0?this._showItemTrigger(!0):(clearTimeout(this._hidingTicket),clearTimeout(this._showingTicket),this._tDom.style.display="none");if(h.length>0){if(this._lastDataIndex!=i||this._lastSeriesIndex!=m[0]){this._lastDataIndex=i,this._lastSeriesIndex=m[0];var b,_;if("function"==typeof a){for(var x=[],y=0,g=h.length;g>y;y++)b=h[y].data[i],_=null!=b?null!=b.value?b.value:b:"-",x.push({seriesIndex:m[y],seriesName:h[y].name||"",series:h[y],dataIndex:i,data:b,name:n.getNameByIndex(i),value:_,0:h[y].name||"",1:n.getNameByIndex(i),2:_,3:b});this._curTicket="axis:"+i,this._tDom.innerHTML=a.call(this.myChart,x,this._curTicket,this._setContent)}else if("string"==typeof a){this._curTicket=0/0,a=a.replace("{a}","{a0}").replace("{b}","{b0}").replace("{c}","{c0}");for(var y=0,g=h.length;g>y;y++)a=a.replace("{a"+y+"}",this._encodeHTML(h[y].name||"")),a=a.replace("{b"+y+"}",this._encodeHTML(n.getNameByIndex(i))),b=h[y].data[i],b=null!=b?null!=b.value?b.value:b:"-",a=a.replace("{c"+y+"}",b instanceof Array?b:this.numAddCommas(b));this._tDom.innerHTML=a}else{this._curTicket=0/0,a=this._encodeHTML(n.getNameByIndex(i));for(var y=0,g=h.length;g>y;y++)a+="
"+this._encodeHTML(h[y].name||"")+" : ",b=h[y].data[i],b=null!=b?null!=b.value?b.value:b:"-",a+=b instanceof Array?b:this.numAddCommas(b);this._tDom.innerHTML=a}}if(r===!1||!this.option.tooltip.showContent)return;this.hasAppend||(this._tDom.style.left=this._zrWidth/2+"px",this._tDom.style.top=this._zrHeight/2+"px",this.dom.firstChild.appendChild(this._tDom),this.hasAppend=!0),this._show(o,p+10,u+10,c)}},_showPolarTrigger:function(e,t){if(null==this.component.polar||null==e||null==t||0>t)return!1;var i,n,a,o=this.option.series,r=[],s=[],l="";if("axis"===this.option.tooltip.trigger){if(!this.option.tooltip.show)return!1;i=this.option.tooltip.formatter,n=this.option.tooltip.position}for(var h=this.option.polar[e].indicator[t].text,m=0,c=o.length;c>m;m++)this._isSelected(o[m].name)&&o[m].polarIndex===e&&"axis"===this.deepQuery([o[m],this.option],"tooltip.trigger")&&(a=this.query(o[m],"tooltip.showContent")||a,i=this.query(o[m],"tooltip.formatter")||i,n=this.query(o[m],"tooltip.position")||n,l+=this._style(this.query(o[m],"tooltip")),r.push(o[m]),s.push(m));if(r.length>0){for(var p,u,V,U=[],m=0,c=r.length;c>m;m++){p=r[m].data;for(var y=0,g=p.length;g>y;y++)u=p[y],this._isSelected(u.name)&&(u=null!=u?u:{name:"",value:{dataIndex:"-"}},V=null!=u.value[t].value?u.value[t].value:u.value[t],U.push({seriesIndex:s[m],seriesName:r[m].name||"",series:r[m],dataIndex:t,data:u,name:u.name,indicator:h,value:V,0:r[m].name||"",1:u.name,2:V,3:h}))}if(U.length<=0)return;if(this._lastDataIndex!=t||this._lastSeriesIndex!=s[0])if(this._lastDataIndex=t,this._lastSeriesIndex=s[0],"function"==typeof i)this._curTicket="axis:"+t,this._tDom.innerHTML=i.call(this.myChart,U,this._curTicket,this._setContent);else if("string"==typeof i){i=i.replace("{a}","{a0}").replace("{b}","{b0}").replace("{c}","{c0}").replace("{d}","{d0}");for(var m=0,c=U.length;c>m;m++)i=i.replace("{a"+m+"}",this._encodeHTML(U[m].seriesName)),i=i.replace("{b"+m+"}",this._encodeHTML(U[m].name)),i=i.replace("{c"+m+"}",this.numAddCommas(U[m].value)),i=i.replace("{d"+m+"}",this._encodeHTML(U[m].indicator));this._tDom.innerHTML=i}else{i=this._encodeHTML(U[0].name)+"
"+this._encodeHTML(U[0].indicator)+" : "+this.numAddCommas(U[0].value);for(var m=1,c=U.length;c>m;m++)i+="
"+this._encodeHTML(U[m].name)+"
",i+=this._encodeHTML(U[m].indicator)+" : "+this.numAddCommas(U[m].value);this._tDom.innerHTML=i}if(a===!1||!this.option.tooltip.showContent)return;return this.hasAppend||(this._tDom.style.left=this._zrWidth/2+"px",this._tDom.style.top=this._zrHeight/2+"px",this.dom.firstChild.appendChild(this._tDom),this.hasAppend=!0),this._show(n,d.getX(this._event),d.getY(this._event),l),!0}},_showItemTrigger:function(e){if(this._curTarget){var t,i,n,a=l.get(this._curTarget,"series"),o=l.get(this._curTarget,"seriesIndex"),r=l.get(this._curTarget,"data"),h=l.get(this._curTarget,"dataIndex"),m=l.get(this._curTarget,"name"),c=l.get(this._curTarget,"value"),p=l.get(this._curTarget,"special"),u=l.get(this._curTarget,"special2"),V="";if("island"!=this._curTarget._type){var U=e?"axis":"item";this.option.tooltip.trigger===U&&(t=this.option.tooltip.formatter,i=this.option.tooltip.position),this.query(a,"tooltip.trigger")===U&&(n=this.query(a,"tooltip.showContent")||n,t=this.query(a,"tooltip.formatter")||t,i=this.query(a,"tooltip.position")||i,V+=this._style(this.query(a,"tooltip"))),n=this.query(r,"tooltip.showContent")||n,t=this.query(r,"tooltip.formatter")||t,i=this.query(r,"tooltip.position")||i,V+=this._style(this.query(r,"tooltip"))}else this._lastItemTriggerId=0/0,n=this.deepQuery([r,a,this.option],"tooltip.showContent"),t=this.deepQuery([r,a,this.option],"tooltip.islandFormatter"),i=this.deepQuery([r,a,this.option],"tooltip.islandPosition");this._lastItemTriggerId!==this._curTarget.id&&(this._lastItemTriggerId=this._curTarget.id,"function"==typeof t?(this._curTicket=(a.name||"")+":"+h,this._tDom.innerHTML=t.call(this.myChart,{seriesIndex:o,seriesName:a.name||"",series:a,dataIndex:h,data:r,name:m,value:c,percent:p,indicator:p,value2:u,indicator2:u,0:a.name||"",1:m,2:c,3:p,4:u,5:r,6:o,7:h},this._curTicket,this._setContent)):"string"==typeof t?(this._curTicket=0/0,t=t.replace("{a}","{a0}").replace("{b}","{b0}").replace("{c}","{c0}"),t=t.replace("{a0}",this._encodeHTML(a.name||"")).replace("{b0}",this._encodeHTML(m)).replace("{c0}",c instanceof Array?c:this.numAddCommas(c)),t=t.replace("{d}","{d0}").replace("{d0}",p||""),t=t.replace("{e}","{e0}").replace("{e0}",l.get(this._curTarget,"special2")||""),this._tDom.innerHTML=t):(this._curTicket=0/0,this._tDom.innerHTML=a.type===s.CHART_TYPE_RADAR&&p?this._itemFormatter.radar.call(this,a,m,c,p):a.type===s.CHART_TYPE_EVENTRIVER?this._itemFormatter.eventRiver.call(this,a,m,c,r):""+(null!=a.name?this._encodeHTML(a.name)+"
":"")+(""===m?"":this._encodeHTML(m)+" : ")+(c instanceof Array?c:this.numAddCommas(c)))),this._axisLineShape.invisible&&this._axisShadowShape.invisible||(this._axisLineShape.invisible=!0,this.zr.modShape(this._axisLineShape.id),this._axisShadowShape.invisible=!0,this.zr.modShape(this._axisShadowShape.id),this.zr.refresh()),n!==!1&&this.option.tooltip.showContent&&(this.hasAppend||(this._tDom.style.left=this._zrWidth/2+"px",this._tDom.style.top=this._zrHeight/2+"px",this.dom.firstChild.appendChild(this._tDom),this.hasAppend=!0),this._show(i,d.getX(this._event)+20,d.getY(this._event)-20,V))}},_itemFormatter:{radar:function(e,t,i,n){var a="";a+=this._encodeHTML(""===t?e.name||"":t),a+=""===a?"":"
";for(var o=0;o";return a},chord:function(e,t,i,n,a){if(null==a)return this._encodeHTML(t)+" ("+this.numAddCommas(i)+")";var o=this._encodeHTML(t),r=this._encodeHTML(n);return""+(null!=e.name?this._encodeHTML(e.name)+"
":"")+o+" -> "+r+" ("+this.numAddCommas(i)+")
"+r+" -> "+o+" ("+this.numAddCommas(a)+")"},eventRiver:function(e,t,i,n){var a="";a+=this._encodeHTML(""===e.name?"":e.name+" : "),a+=this._encodeHTML(t),a+=""===a?"":"
",n=n.evolution;for(var o=0,r=n.length;r>o;o++)a+='

',n[o].detail&&(n[o].detail.img&&(a+=''),a+='
'+n[o].time+"
",a+='',a+=n[o].detail.text+"
",a+="
");return a}},_styleAxisPointer:function(e,t,i,n,a,o,r,s){if(e.length>0){var l,h,d=this.option.tooltip.axisPointer,m=d.type,c={line:{},cross:{},shadow:{}};for(var p in c)c[p].color=d[p+"Style"].color,c[p].width=d[p+"Style"].width,c[p].type=d[p+"Style"].type;for(var u=0,V=e.length;V>u;u++)"axis"===this.deepQuery([e[u],this.option],"tooltip.trigger")&&(l=e[u],h=this.query(l,"tooltip.axisPointer.type"),m=h||m,h&&(c[h].color=this.query(l,"tooltip.axisPointer."+h+"Style.color")||c[h].color,c[h].width=this.query(l,"tooltip.axisPointer."+h+"Style.width")||c[h].width,c[h].type=this.query(l,"tooltip.axisPointer."+h+"Style.type")||c[h].type));"line"===m?(this._axisLineShape.style={xStart:t,yStart:i,xEnd:n,yEnd:a,strokeColor:c.line.color,lineWidth:c.line.width,lineType:c.line.type},this._axisLineShape.invisible=!1,this.zr.modShape(this._axisLineShape.id)):"cross"===m?(this._axisCrossShape.style={brushType:"stroke",rect:this.component.grid.getArea(),x:r,y:s,text:("( "+this.component.xAxis.getAxis(0).getValueFromCoord(r)+" , "+this.component.yAxis.getAxis(0).getValueFromCoord(s)+" )").replace(" , "," ").replace(" , "," "),textPosition:"specific",strokeColor:c.cross.color,lineWidth:c.cross.width,lineType:c.cross.type},this.component.grid.getXend()-r>100?(this._axisCrossShape.style.textAlign="left",this._axisCrossShape.style.textX=r+10):(this._axisCrossShape.style.textAlign="right",this._axisCrossShape.style.textX=r-10),s-this.component.grid.getY()>50?(this._axisCrossShape.style.textBaseline="bottom",this._axisCrossShape.style.textY=s-10):(this._axisCrossShape.style.textBaseline="top",this._axisCrossShape.style.textY=s+10),this._axisCrossShape.invisible=!1,this.zr.modShape(this._axisCrossShape.id)):"shadow"===m&&((null==c.shadow.width||"auto"===c.shadow.width||isNaN(c.shadow.width))&&(c.shadow.width=o),t===n?Math.abs(this.component.grid.getX()-t)<2?(c.shadow.width/=2,t=n+=c.shadow.width/2):Math.abs(this.component.grid.getXend()-t)<2&&(c.shadow.width/=2,t=n-=c.shadow.width/2):i===a&&(Math.abs(this.component.grid.getY()-i)<2?(c.shadow.width/=2,i=a+=c.shadow.width/2):Math.abs(this.component.grid.getYend()-i)<2&&(c.shadow.width/=2,i=a-=c.shadow.width/2)),this._axisShadowShape.style={xStart:t,yStart:i,xEnd:n,yEnd:a,strokeColor:c.shadow.color,lineWidth:c.shadow.width},this._axisShadowShape.invisible=!1,this.zr.modShape(this._axisShadowShape.id)),this.zr.refresh()}},__onmousemove:function(e){if(clearTimeout(this._hidingTicket),clearTimeout(this._showingTicket),!this._mousein||!this._enterable){var t=e.target,i=d.getX(e.event),n=d.getY(e.event);if(t){this._curTarget=t,this._event=e.event,this._event.zrenderX=i,this._event.zrenderY=n;var a;if(this._needAxisTrigger&&this.component.polar&&-1!=(a=this.component.polar.isInside([i,n])))for(var o=this.option.series,l=0,h=o.length;h>l;l++)if(o[l].polarIndex===a&&"axis"===this.deepQuery([o[l],this.option],"tooltip.trigger")){this._curTarget=null;break}this._showingTicket=setTimeout(this._tryShow,this._showDelay)}else this._curTarget=!1,this._event=e.event,this._event.zrenderX=i,this._event.zrenderY=n,this._needAxisTrigger&&this.component.grid&&m.isInside(r,this.component.grid.getArea(),i,n)?this._showingTicket=setTimeout(this._tryShow,this._showDelay):this._needAxisTrigger&&this.component.polar&&-1!=this.component.polar.isInside([i,n])?this._showingTicket=setTimeout(this._tryShow,this._showDelay):(!this._event.connectTrigger&&this.messageCenter.dispatch(s.EVENT.TOOLTIP_OUT_GRID,this._event,null,this.myChart),this._hidingTicket=setTimeout(this._hide,this._hideDelay))}},__onglobalout:function(){clearTimeout(this._hidingTicket),clearTimeout(this._showingTicket),this._hidingTicket=setTimeout(this._hide,this._hideDelay)},__setContent:function(e,t){this._tDom&&(e===this._curTicket&&(this._tDom.innerHTML=t),setTimeout(this._refixed,20))},ontooltipHover:function(e,t){if(!this._lastTipShape||this._lastTipShape&&this._lastTipShape.dataIndex!=e.dataIndex){this._lastTipShape&&this._lastTipShape.tipShape.length>0&&(this.zr.delShape(this._lastTipShape.tipShape),this.shapeList.length=2);for(var i=0,n=t.length;n>i;i++)t[i].zlevel=this._zlevelBase,t[i].style=u.prototype.getHighlightStyle(t[i].style,t[i].highlightStyle),t[i].draggable=!1,t[i].hoverable=!1,t[i].clickable=!1,t[i].ondragend=null,t[i].ondragover=null,t[i].ondrop=null,this.shapeList.push(t[i]),this.zr.addShape(t[i]);this._lastTipShape={dataIndex:e.dataIndex,tipShape:t}}},ondragend:function(){this._hide()},onlegendSelected:function(e){this._selectedMap=e.selected},_setSelectedMap:function(){this._selectedMap=this.component.legend?p.clone(this.component.legend.getSelectedMap()):{}},_isSelected:function(e){return null!=this._selectedMap[e]?this._selectedMap[e]:!0},showTip:function(e){if(e){var t,i=this.option.series;if(null!=e.seriesIndex)t=e.seriesIndex;else for(var n=e.seriesName,a=0,o=i.length;o>a;a++)if(i[a].name===n){t=a;break}var r=i[t];if(null!=r){var d=this.myChart.chart[r.type],m="axis"===this.deepQuery([r,this.option],"tooltip.trigger");if(d)if(m){var c=e.dataIndex;switch(d.type){case s.CHART_TYPE_LINE:case s.CHART_TYPE_BAR:case s.CHART_TYPE_K:if(null==this.component.xAxis||null==this.component.yAxis||r.data.length<=c)return;var p=r.xAxisIndex||0,u=r.yAxisIndex||0;this._event=this.component.xAxis.getAxis(p).type===s.COMPONENT_TYPE_AXIS_CATEGORY?{zrenderX:this.component.xAxis.getAxis(p).getCoordByIndex(c),zrenderY:this.component.grid.getY()+(this.component.grid.getYend()-this.component.grid.getY())/4}:{zrenderX:this.component.grid.getX()+(this.component.grid.getXend()-this.component.grid.getX())/4,zrenderY:this.component.yAxis.getAxis(u).getCoordByIndex(c)},this._showAxisTrigger(p,u,c);break;case s.CHART_TYPE_RADAR:if(null==this.component.polar||r.data[0].value.length<=c)return;var V=r.polarIndex||0,U=this.component.polar.getVector(V,c,"max");this._event={zrenderX:U[0],zrenderY:U[1]},this._showPolarTrigger(V,c)}}else{var y,g,f=d.shapeList;switch(d.type){case s.CHART_TYPE_LINE:case s.CHART_TYPE_BAR:case s.CHART_TYPE_K:case s.CHART_TYPE_SCATTER:for(var c=e.dataIndex,a=0,o=f.length;o>a;a++)if(l.get(f[a],"seriesIndex")==t&&l.get(f[a],"dataIndex")==c){this._curTarget=f[a],y=f[a].style.x,g=d.type!=s.CHART_TYPE_K?f[a].style.y:f[a].style.y[0];break}break;case s.CHART_TYPE_RADAR:for(var c=e.dataIndex,a=0,o=f.length;o>a;a++)if("polygon"===f[a].type&&l.get(f[a],"seriesIndex")==t&&l.get(f[a],"dataIndex")==c){this._curTarget=f[a];var U=this.component.polar.getCenter(r.polarIndex||0);y=U[0],g=U[1];break}break;case s.CHART_TYPE_PIE:for(var b=e.name,a=0,o=f.length;o>a;a++)if("sector"===f[a].type&&l.get(f[a],"seriesIndex")==t&&l.get(f[a],"name")==b){this._curTarget=f[a];var _=this._curTarget.style,x=(_.startAngle+_.endAngle)/2*Math.PI/180;y=this._curTarget.style.x+Math.cos(x)*_.r/1.5,g=this._curTarget.style.y-Math.sin(x)*_.r/1.5;break}break;case s.CHART_TYPE_MAP:for(var b=e.name,k=r.mapType,a=0,o=f.length;o>a;a++)if("text"===f[a].type&&f[a]._mapType===k&&f[a].style._name===b){this._curTarget=f[a],y=this._curTarget.style.x+this._curTarget.position[0],g=this._curTarget.style.y+this._curTarget.position[1];break}break;case s.CHART_TYPE_CHORD:for(var b=e.name,a=0,o=f.length;o>a;a++)if("sector"===f[a].type&&l.get(f[a],"name")==b){this._curTarget=f[a];var _=this._curTarget.style,x=(_.startAngle+_.endAngle)/2*Math.PI/180;return y=this._curTarget.style.x+Math.cos(x)*(_.r-2),g=this._curTarget.style.y-Math.sin(x)*(_.r-2),void this.zr.trigger(h.EVENT.MOUSEMOVE,{zrenderX:y,zrenderY:g})}break;case s.CHART_TYPE_FORCE:for(var b=e.name,a=0,o=f.length;o>a;a++)if("circle"===f[a].type&&l.get(f[a],"name")==b){this._curTarget=f[a],y=this._curTarget.position[0],g=this._curTarget.position[1];break}}null!=y&&null!=g&&(this._event={zrenderX:y,zrenderY:g},this.zr.addHoverShape(this._curTarget),this.zr.refreshHover(),this._showItemTrigger())}}}},hideTip:function(){this._hide()},refresh:function(e){if(this._zrHeight=this.zr.getHeight(),this._zrWidth=this.zr.getWidth(),this._lastTipShape&&this._lastTipShape.tipShape.length>0&&this.zr.delShape(this._lastTipShape.tipShape),this._lastTipShape=!1,this.shapeList.length=2,this._lastDataIndex=-1,this._lastSeriesIndex=-1,this._lastItemTriggerId=-1,e){this.option=e,this.option.tooltip=this.reformOption(this.option.tooltip),this.option.tooltip.textStyle=p.merge(this.option.tooltip.textStyle,this.ecTheme.textStyle),this._needAxisTrigger=!1,"axis"===this.option.tooltip.trigger&&(this._needAxisTrigger=!0);for(var t=this.option.series,i=0,n=t.length;n>i;i++)if("axis"===this.query(t[i],"tooltip.trigger")){this._needAxisTrigger=!0;break}this._showDelay=this.option.tooltip.showDelay,this._hideDelay=this.option.tooltip.hideDelay,this._defaultCssText=this._style(this.option.tooltip),this._setSelectedMap(),this._axisLineWidth=this.option.tooltip.axisPointer.lineStyle.width,this._enterable=this.option.tooltip.enterable}if(this.showing){var a=this;setTimeout(function(){a.zr.trigger(h.EVENT.MOUSEMOVE,a.zr.handler._event)},50)}},onbeforDispose:function(){this._lastTipShape&&this._lastTipShape.tipShape.length>0&&this.zr.delShape(this._lastTipShape.tipShape),clearTimeout(this._hidingTicket),clearTimeout(this._showingTicket),this.zr.un(h.EVENT.MOUSEMOVE,this._onmousemove),this.zr.un(h.EVENT.GLOBALOUT,this._onglobalout),this.hasAppend&&this.dom.firstChild.removeChild(this._tDom),this._tDom=null +},_encodeHTML:function(e){return String(e).replace(/&/g,"&").replace(//g,">").replace(/"/g,""").replace(/'/g,"'")}},p.inherits(t,i),e("../component").define("tooltip",t),t}),define("echarts/component/legend",["require","./base","zrender/shape/Text","zrender/shape/Rectangle","zrender/shape/Sector","../util/shape/Icon","../util/shape/Candle","../config","zrender/tool/util","zrender/tool/area","../component"],function(e){function t(e,t,n,a,o){if(!this.query(a,"legend.data"))return void console.error("option.legend.data has not been defined.");i.call(this,e,t,n,a,o);var r=this;r._legendSelected=function(e){r.__legendSelected(e)},r._dispatchHoverLink=function(e){return r.__dispatchHoverLink(e)},this._colorIndex=0,this._colorMap={},this._selectedMap={},this._hasDataMap={},this.refresh(a)}var i=e("./base"),n=e("zrender/shape/Text"),a=e("zrender/shape/Rectangle"),o=e("zrender/shape/Sector"),r=e("../util/shape/Icon"),s=e("../util/shape/Candle"),l=e("../config"),h=e("zrender/tool/util"),d=e("zrender/tool/area");t.prototype={type:l.COMPONENT_TYPE_LEGEND,_buildShape:function(){if(this.legendOption.show){this._itemGroupLocation=this._getItemGroupLocation(),this._buildBackground(),this._buildItem();for(var e=0,t=this.shapeList.length;t>e;e++)this.zr.addShape(this.shapeList[e])}},_buildItem:function(){var e,t,i,a,o,s,l,m,c=this.legendOption.data,p=c.length,u=this.legendOption.textStyle,V=this.zr.getWidth(),U=this.zr.getHeight(),y=this._itemGroupLocation.x,g=this._itemGroupLocation.y,f=this.legendOption.itemWidth,b=this.legendOption.itemHeight,_=this.legendOption.itemGap;"vertical"===this.legendOption.orient&&"right"===this.legendOption.x&&(y=this._itemGroupLocation.x+this._itemGroupLocation.width-f);for(var x=0;p>x;x++)o=h.merge(c[x].textStyle||{},u),s=this.getFont(o),e=this._getName(c[x]),l=this._getFormatterName(e),""!==e?(t=c[x].icon||this._getSomethingByName(e).type,m=this.getColor(e),"horizontal"===this.legendOption.orient?200>V-y&&f+5+d.getTextWidth(l,s)+(x===p-1||""===c[x+1]?0:_)>=V-y&&(y=this._itemGroupLocation.x,g+=b+_):200>U-g&&b+(x===p-1||""===c[x+1]?0:_)>=U-g&&("right"===this.legendOption.x?y-=this._itemGroupLocation.maxWidth+_:y+=this._itemGroupLocation.maxWidth+_,g=this._itemGroupLocation.y),i=this._getItemShapeByType(y,g,f,b,this._selectedMap[e]&&this._hasDataMap[e]?m:"#ccc",t,m),i._name=e,i=new r(i),a={zlevel:this._zlevelBase,style:{x:y+f+5,y:g+b/2,color:this._selectedMap[e]?"auto"===o.color?m:o.color:"#ccc",text:l,textFont:s,textBaseline:"middle"},highlightStyle:{color:m,brushType:"fill"},hoverable:!!this.legendOption.selectedMode,clickable:!!this.legendOption.selectedMode},"vertical"===this.legendOption.orient&&"right"===this.legendOption.x&&(a.style.x-=f+10,a.style.textAlign="right"),a._name=e,a=new n(a),this.legendOption.selectedMode&&(i.onclick=a.onclick=this._legendSelected,i.onmouseover=a.onmouseover=this._dispatchHoverLink,i.hoverConnect=a.id,a.hoverConnect=i.id),this.shapeList.push(i),this.shapeList.push(a),"horizontal"===this.legendOption.orient?y+=f+5+d.getTextWidth(l,s)+_:g+=b+_):"horizontal"===this.legendOption.orient?(y=this._itemGroupLocation.x,g+=b+_):("right"===this.legendOption.x?y-=this._itemGroupLocation.maxWidth+_:y+=this._itemGroupLocation.maxWidth+_,g=this._itemGroupLocation.y);"horizontal"===this.legendOption.orient&&"center"===this.legendOption.x&&g!=this._itemGroupLocation.y&&this._mLineOptimize()},_getName:function(e){return"undefined"!=typeof e.name?e.name:e},_getFormatterName:function(e){var t,i=this.legendOption.formatter;return t="function"==typeof i?i.call(this.myChart,e):"string"==typeof i?i.replace("{name}",e):e},_getFormatterNameFromData:function(e){var t=this._getName(e);return this._getFormatterName(t)},_mLineOptimize:function(){for(var e=[],t=this._itemGroupLocation.x,i=2,n=this.shapeList.length;n>i;i++)this.shapeList[i].style.x===t?e.push((this._itemGroupLocation.width-(this.shapeList[i-1].style.x+d.getTextWidth(this.shapeList[i-1].style.text,this.shapeList[i-1].style.textFont)-t))/2):i===n-1&&e.push((this._itemGroupLocation.width-(this.shapeList[i].style.x+d.getTextWidth(this.shapeList[i].style.text,this.shapeList[i].style.textFont)-t))/2);for(var a=-1,i=1,n=this.shapeList.length;n>i;i++)this.shapeList[i].style.x===t&&a++,0!==e[a]&&(this.shapeList[i].style.x+=e[a])},_buildBackground:function(){var e=this.reformCssArray(this.legendOption.padding);this.shapeList.push(new a({zlevel:this._zlevelBase,hoverable:!1,style:{x:this._itemGroupLocation.x-e[3],y:this._itemGroupLocation.y-e[0],width:this._itemGroupLocation.width+e[3]+e[1],height:this._itemGroupLocation.height+e[0]+e[2],brushType:0===this.legendOption.borderWidth?"fill":"both",color:this.legendOption.backgroundColor,strokeColor:this.legendOption.borderColor,lineWidth:this.legendOption.borderWidth}}))},_getItemGroupLocation:function(){var e=this.legendOption.data,t=e.length,i=this.legendOption.itemGap,n=this.legendOption.itemWidth+5,a=this.legendOption.itemHeight,o=this.legendOption.textStyle,r=this.getFont(o),s=0,l=0,m=this.reformCssArray(this.legendOption.padding),c=this.zr.getWidth()-m[1]-m[3],p=this.zr.getHeight()-m[0]-m[2],u=0,V=0;if("horizontal"===this.legendOption.orient){l=a;for(var U=0;t>U;U++)""!==this._getName(e[U])?u+=n+d.getTextWidth(this._getFormatterNameFromData(e[U]),e[U].textStyle?this.getFont(h.merge(e[U].textStyle||{},o)):r)+i:(u-=i,u>c?(s=c,l+=a+i):s=Math.max(s,u),l+=a+i,u=0);l=Math.max(l,a),u-=i,u>c?(s=c,l+=a+i):s=Math.max(s,u)}else{for(var U=0;t>U;U++)V=Math.max(V,d.getTextWidth(this._getFormatterNameFromData(e[U]),e[U].textStyle?this.getFont(h.merge(e[U].textStyle||{},o)):r));V+=n,s=V;for(var U=0;t>U;U++)""!==this._getName(e[U])?u+=a+i:(u-=i,u>p?(l=p,s+=V+i):l=Math.max(l,u),s+=V+i,u=0);s=Math.max(s,V),u-=i,u>p?(l=p,s+=V+i):l=Math.max(l,u)}c=this.zr.getWidth(),p=this.zr.getHeight();var y;switch(this.legendOption.x){case"center":y=Math.floor((c-s)/2);break;case"left":y=m[3]+this.legendOption.borderWidth;break;case"right":y=c-s-m[1]-m[3]-2*this.legendOption.borderWidth;break;default:y=this.parsePercent(this.legendOption.x,c)}var g;switch(this.legendOption.y){case"top":g=m[0]+this.legendOption.borderWidth;break;case"bottom":g=p-l-m[0]-m[2]-2*this.legendOption.borderWidth;break;case"center":g=Math.floor((p-l)/2);break;default:g=this.parsePercent(this.legendOption.y,p)}return{x:y,y:g,width:s,height:l,maxWidth:V}},_getSomethingByName:function(e){for(var t,i=this.option.series,n=0,a=i.length;a>n;n++){if(i[n].name===e)return{type:i[n].type,series:i[n],seriesIndex:n,data:null,dataIndex:-1};if(i[n].type===l.CHART_TYPE_PIE||i[n].type===l.CHART_TYPE_RADAR||i[n].type===l.CHART_TYPE_CHORD||i[n].type===l.CHART_TYPE_FORCE||i[n].type===l.CHART_TYPE_FUNNEL){t=i[n].categories||i[n].data||i[n].nodes;for(var o=0,r=t.length;r>o;o++)if(t[o].name===e)return{type:i[n].type,series:i[n],seriesIndex:n,data:t[o],dataIndex:o}}}return{type:"bar",series:null,seriesIndex:-1,data:null,dataIndex:-1}},_getItemShapeByType:function(e,t,i,n,a,o,r){var s,l="#ccc"===a?r:a,h={zlevel:this._zlevelBase,style:{iconType:"legendicon"+o,x:e,y:t,width:i,height:n,color:a,strokeColor:a,lineWidth:2},highlightStyle:{color:l,strokeColor:l,lineWidth:1},hoverable:this.legendOption.selectedMode,clickable:this.legendOption.selectedMode};if(o.match("image")){var s=o.replace(new RegExp("^image:\\/\\/"),"");o="image"}switch(o){case"line":h.style.brushType="stroke",h.highlightStyle.lineWidth=3;break;case"radar":case"scatter":h.highlightStyle.lineWidth=3;break;case"k":h.style.brushType="both",h.highlightStyle.lineWidth=3,h.highlightStyle.color=h.style.color=this.query(this.ecTheme,"k.itemStyle.normal.color")||"#fff",h.style.strokeColor="#ccc"!=a?this.query(this.ecTheme,"k.itemStyle.normal.lineStyle.color")||"#ff3200":a;break;case"image":h.style.iconType="image",h.style.image=s,"#ccc"===a&&(h.style.opacity=.5)}return h},__legendSelected:function(e){var t=e.target._name;if("single"===this.legendOption.selectedMode)for(var i in this._selectedMap)this._selectedMap[i]=!1;this._selectedMap[t]=!this._selectedMap[t],this.messageCenter.dispatch(l.EVENT.LEGEND_SELECTED,e.event,{selected:this._selectedMap,target:t},this.myChart)},__dispatchHoverLink:function(e){this.messageCenter.dispatch(l.EVENT.LEGEND_HOVERLINK,e.event,{target:e.target._name},this.myChart)},refresh:function(e){if(e){this.option=e||this.option,this.option.legend=this.reformOption(this.option.legend),this.legendOption=this.option.legend;var t,i,n,a,o=this.legendOption.data||[];if(this.legendOption.selected)for(var r in this.legendOption.selected)this._selectedMap[r]="undefined"!=typeof this._selectedMap[r]?this._selectedMap[r]:this.legendOption.selected[r];for(var s=0,h=o.length;h>s;s++)t=this._getName(o[s]),""!==t&&(i=this._getSomethingByName(t),i.series?(this._hasDataMap[t]=!0,a=!i.data||i.type!==l.CHART_TYPE_PIE&&i.type!==l.CHART_TYPE_FORCE&&i.type!==l.CHART_TYPE_FUNNEL?[i.series]:[i.data,i.series],n=this.getItemStyleColor(this.deepQuery(a,"itemStyle.normal.color"),i.seriesIndex,i.dataIndex,i.data),n&&i.type!=l.CHART_TYPE_K&&this.setColor(t,n),this._selectedMap[t]=null!=this._selectedMap[t]?this._selectedMap[t]:!0):this._hasDataMap[t]=!1)}this.clear(),this._buildShape()},getRelatedAmount:function(e){for(var t,i=0,n=this.option.series,a=0,o=n.length;o>a;a++)if(n[a].name===e&&i++,n[a].type===l.CHART_TYPE_PIE||n[a].type===l.CHART_TYPE_RADAR||n[a].type===l.CHART_TYPE_CHORD||n[a].type===l.CHART_TYPE_FORCE||n[a].type===l.CHART_TYPE_FUNNEL){t=n[a].type!=l.CHART_TYPE_FORCE?n[a].data:n[a].categories;for(var r=0,s=t.length;s>r;r++)t[r].name===e&&"-"!=t[r].value&&i++}return i},setColor:function(e,t){this._colorMap[e]=t},getColor:function(e){return this._colorMap[e]||(this._colorMap[e]=this.zr.getColor(this._colorIndex++)),this._colorMap[e]},hasColor:function(e){return this._colorMap[e]?this._colorMap[e]:!1},add:function(e,t){for(var i=this.legendOption.data,n=0,a=i.length;a>n;n++)if(this._getName(i[n])===e)return;this.legendOption.data.push(e),this.setColor(e,t),this._selectedMap[e]=!0,this._hasDataMap[e]=!0},del:function(e){for(var t=this.legendOption.data,i=0,n=t.length;n>i;i++)if(this._getName(t[i])===e)return this.legendOption.data.splice(i,1)},getItemShape:function(e){if(null!=e)for(var t,i=0,n=this.shapeList.length;n>i;i++)if(t=this.shapeList[i],t._name===e&&"text"!=t.type)return t},setItemShape:function(e,t){for(var i,n=0,a=this.shapeList.length;a>n;n++)i=this.shapeList[n],i._name===e&&"text"!=i.type&&(this._selectedMap[e]||(t.style.color="#ccc",t.style.strokeColor="#ccc"),this.zr.modShape(i.id,t))},isSelected:function(e){return"undefined"!=typeof this._selectedMap[e]?this._selectedMap[e]:!0},getSelectedMap:function(){return this._selectedMap},setSelected:function(e,t){if("single"===this.legendOption.selectedMode)for(var i in this._selectedMap)this._selectedMap[i]=!1;this._selectedMap[e]=t,this.messageCenter.dispatch(l.EVENT.LEGEND_SELECTED,null,{selected:this._selectedMap,target:e},this.myChart)},onlegendSelected:function(e,t){var i=e.selected;for(var n in i)this._selectedMap[n]!=i[n]&&(t.needRefresh=!0),this._selectedMap[n]=i[n]}};var m={line:function(e,t){var i=t.height/2;e.moveTo(t.x,t.y+i),e.lineTo(t.x+t.width,t.y+i)},pie:function(e,t){var i=t.x,n=t.y,a=t.width,r=t.height;o.prototype.buildPath(e,{x:i+a/2,y:n+r+2,r:r+2,r0:6,startAngle:45,endAngle:135})},eventRiver:function(e,t){var i=t.x,n=t.y,a=t.width,o=t.height;e.moveTo(i,n+o),e.bezierCurveTo(i+a,n+o,i,n+4,i+a,n+4),e.lineTo(i+a,n),e.bezierCurveTo(i,n,i+a,n+o-4,i,n+o-4),e.lineTo(i,n+o)},k:function(e,t){var i=t.x,n=t.y,a=t.width,o=t.height;s.prototype.buildPath(e,{x:i+a/2,y:[n+1,n+1,n+o-6,n+o],width:a-6})},bar:function(e,t){var i=t.x,n=t.y+1,a=t.width,o=t.height-2,r=3;e.moveTo(i+r,n),e.lineTo(i+a-r,n),e.quadraticCurveTo(i+a,n,i+a,n+r),e.lineTo(i+a,n+o-r),e.quadraticCurveTo(i+a,n+o,i+a-r,n+o),e.lineTo(i+r,n+o),e.quadraticCurveTo(i,n+o,i,n+o-r),e.lineTo(i,n+r),e.quadraticCurveTo(i,n,i+r,n)},force:function(e,t){r.prototype.iconLibrary.circle(e,t)},radar:function(e,t){var i=6,n=t.x+t.width/2,a=t.y+t.height/2,o=t.height/2,r=2*Math.PI/i,s=-Math.PI/2,l=n+o*Math.cos(s),h=a+o*Math.sin(s);e.moveTo(l,h),s+=r;for(var d=0,m=i-1;m>d;d++)e.lineTo(n+o*Math.cos(s),a+o*Math.sin(s)),s+=r;e.lineTo(l,h)}};m.chord=m.pie,m.map=m.bar;for(var c in m)r.prototype.iconLibrary["legendicon"+c]=m[c];return h.inherits(t,i),e("../component").define("legend",t),t}),define("echarts/util/ecData",[],function(){function e(e,t,i,n,a,o,r,s){var l;return"undefined"!=typeof n&&(l=null==n.value?n:n.value),e._echartsData={_series:t,_seriesIndex:i,_data:n,_dataIndex:a,_name:o,_value:l,_special:r,_special2:s},e._echartsData}function t(e,t){var i=e._echartsData;if(!t)return i;switch(t){case"series":case"seriesIndex":case"data":case"dataIndex":case"name":case"value":case"special":case"special2":return i&&i["_"+t]}return null}function i(e,t,i){switch(e._echartsData=e._echartsData||{},t){case"series":case"seriesIndex":case"data":case"dataIndex":case"name":case"value":case"special":case"special2":e._echartsData["_"+t]=i}}function n(e,t){t._echartsData={_series:e._echartsData._series,_seriesIndex:e._echartsData._seriesIndex,_data:e._echartsData._data,_dataIndex:e._echartsData._dataIndex,_name:e._echartsData._name,_value:e._echartsData._value,_special:e._echartsData._special,_special2:e._echartsData._special2}}return{pack:e,set:i,get:t,clone:n}}),define("echarts/chart",[],function(){var e={},t={};return e.define=function(i,n){return t[i]=n,e},e.get=function(e){return t[e]},e}),define("zrender/tool/color",["require","../tool/util"],function(e){function t(e){D=e}function i(){D=N}function n(e,t){return e=0|e,t=t||D,t[e%t.length]}function a(e){H=e}function o(){B=H}function r(){return H}function s(e,t,i,n,a,o,r){O||(O=P.getContext());for(var s=O.createRadialGradient(e,t,i,n,a,o),l=0,h=r.length;h>l;l++)s.addColorStop(r[l][0],r[l][1]);return s.__nonRecursion=!0,s}function l(e,t,i,n,a){O||(O=P.getContext());for(var o=O.createLinearGradient(e,t,i,n),r=0,s=a.length;s>r;r++)o.addColorStop(a[r][0],a[r][1]);return o.__nonRecursion=!0,o}function h(e,t,i){e=u(e),t=u(t),e=K(e),t=K(t);for(var n=[],a=(t[0]-e[0])/i,o=(t[1]-e[1])/i,r=(t[2]-e[2])/i,s=(t[3]-e[3])/i,l=0,h=e[0],d=e[1],c=e[2],p=e[3];i>l;l++)n[l]=m([C(Math.floor(h),[0,255]),C(Math.floor(d),[0,255]),C(Math.floor(c),[0,255]),p.toFixed(4)-0],"rgba"),h+=a,d+=o,c+=r,p+=s;return h=t[0],d=t[1],c=t[2],p=t[3],n[l]=m([h,d,c,p],"rgba"),n}function d(e,t){var i=[],n=e.length;if(void 0===t&&(t=20),1===n)i=h(e[0],e[0],t);else if(n>1)for(var a=0,o=n-1;o>a;a++){var r=h(e[a],e[a+1],t);o-1>a&&r.pop(),i=i.concat(r)}return i}function m(e,t){if(t=t||"rgb",e&&(3===e.length||4===e.length)){if(e=T(e,function(e){return e>1?Math.ceil(e):e}),t.indexOf("hex")>-1)return"#"+((1<<24)+(e[0]<<16)+(e[1]<<8)+ +e[2]).toString(16).slice(1);if(t.indexOf("hs")>-1){var i=T(e.slice(1,3),function(e){return e+"%"});e[1]=i[0],e[2]=i[1]}return t.indexOf("a")>-1?(3===e.length&&e.push(1),e[3]=C(e[3],[0,1]),t+"("+e.slice(0,4).join(",")+")"):t+"("+e.slice(0,3).join(",")+")"}}function c(e){e=L(e),e.indexOf("rgba")<0&&(e=u(e));var t=[],i=0;return e.replace(/[\d.]+/g,function(e){e=3>i?0|e:+e,t[i++]=e}),t}function p(e,t){if(!E(e))return e;var i=K(e),n=i[3];return"undefined"==typeof n&&(n=1),e.indexOf("hsb")>-1?i=A(i):e.indexOf("hsl")>-1&&(i=z(i)),t.indexOf("hsb")>-1||t.indexOf("hsv")>-1?i=J(i):t.indexOf("hsl")>-1&&(i=F(i)),i[3]=n,m(i,t)}function u(e){return p(e,"rgba")}function V(e){return p(e,"rgb")}function U(e){return p(e,"hex")}function y(e){return p(e,"hsva")}function g(e){return p(e,"hsv")}function f(e){return p(e,"hsba")}function b(e){return p(e,"hsb")}function _(e){return p(e,"hsla")}function x(e){return p(e,"hsl")}function k(e){for(var t in G)if(U(G[t])===U(e))return t;return null}function L(e){return String(e).replace(/\s+/g,"")}function v(e){if(G[e]&&(e=G[e]),e=L(e),e=e.replace(/hsv/i,"hsb"),/^#[\da-f]{3}$/i.test(e)){e=parseInt(e.slice(1),16);var t=(3840&e)<<8,i=(240&e)<<4,n=15&e;e="#"+((1<<24)+(t<<4)+t+(i<<4)+i+(n<<4)+n).toString(16).slice(1)}return e}function W(e,t){if(!E(e))return e;var i=t>0?1:-1;"undefined"==typeof t&&(t=0),t=Math.abs(t)>1?1:Math.abs(t),e=V(e);for(var n=K(e),a=0;3>a;a++)n[a]=1===i?n[a]*(1-t)|0:(255-n[a])*t+n[a]|0;return"rgb("+n.join(",")+")"}function w(e){if(!E(e))return e;var t=K(u(e));return t=T(t,function(e){return 255-e}),m(t,"rgb")}function X(e,t,i){if(!E(e)||!E(t))return e;"undefined"==typeof i&&(i=.5),i=1-C(i,[0,1]);for(var n=2*i-1,a=K(u(e)),o=K(u(t)),r=a[3]-o[3],s=((n*r===-1?n:(n+r)/(1+n*r))+1)/2,l=1-s,h=[],d=0;3>d;d++)h[d]=a[d]*s+o[d]*l;var c=a[3]*i+o[3]*(1-i);return c=Math.max(0,Math.min(1,c)),1===a[3]&&1===o[3]?m(h,"rgb"):(h[3]=c,m(h,"rgba"))}function I(){return"#"+(Math.random().toString(16)+"0000").slice(2,8)}function K(e){e=v(e);var t=e.match(R);if(null===t)throw new Error("The color format error");var i,n,a,o=[];if(t[2])i=t[2].replace("#","").split(""),a=[i[0]+i[1],i[2]+i[3],i[4]+i[5]],o=T(a,function(e){return C(parseInt(e,16),[0,255])});else if(t[4]){var r=t[4].split(",");n=r[3],a=r.slice(0,3),o=T(a,function(e){return e=Math.floor(e.indexOf("%")>0?2.55*parseInt(e,0):e),C(e,[0,255])}),"undefined"!=typeof n&&o.push(C(parseFloat(n),[0,1]))}else if(t[5]||t[6]){var s=(t[5]||t[6]).split(","),l=parseInt(s[0],0)/360,h=s[1],d=s[2];n=s[3],o=T([h,d],function(e){return C(parseFloat(e)/100,[0,1])}),o.unshift(l),"undefined"!=typeof n&&o.push(C(parseFloat(n),[0,1]))}return o}function S(e,t){if(!E(e))return e;null===t&&(t=1);var i=K(u(e));return i[3]=C(Number(t).toFixed(4),[0,1]),m(i,"rgba")}function T(e,t){if("function"!=typeof t)throw new TypeError;for(var i=e?e.length:0,n=0;i>n;n++)e[n]=t(e[n]);return e}function C(e,t){return e<=t[0]?e=t[0]:e>=t[1]&&(e=t[1]),e}function E(e){return e instanceof Array||"string"==typeof e}function A(e){var t,i,n,a=e[0],o=e[1],r=e[2];if(0===o)t=255*r,i=255*r,n=255*r;else{var s=6*a;6===s&&(s=0);var l=0|s,h=r*(1-o),d=r*(1-o*(s-l)),m=r*(1-o*(1-(s-l))),c=0,p=0,u=0;0===l?(c=r,p=m,u=h):1===l?(c=d,p=r,u=h):2===l?(c=h,p=r,u=m):3===l?(c=h,p=d,u=r):4===l?(c=m,p=h,u=r):(c=r,p=h,u=d),t=255*c,i=255*p,n=255*u}return[t,i,n]}function z(e){var t,i,n,a=e[0],o=e[1],r=e[2];if(0===o)t=255*r,i=255*r,n=255*r;else{var s;s=.5>r?r*(1+o):r+o-o*r;var l=2*r-s;t=255*M(l,s,a+1/3),i=255*M(l,s,a),n=255*M(l,s,a-1/3)}return[t,i,n]}function M(e,t,i){return 0>i&&(i+=1),i>1&&(i-=1),1>6*i?e+6*(t-e)*i:1>2*i?t:2>3*i?e+(t-e)*(2/3-i)*6:e}function J(e){var t,i,n=e[0]/255,a=e[1]/255,o=e[2]/255,r=Math.min(n,a,o),s=Math.max(n,a,o),l=s-r,h=s;if(0===l)t=0,i=0;else{i=l/s;var d=((s-n)/6+l/2)/l,m=((s-a)/6+l/2)/l,c=((s-o)/6+l/2)/l;n===s?t=c-m:a===s?t=1/3+d-c:o===s&&(t=2/3+m-d),0>t&&(t+=1),t>1&&(t-=1)}return t=360*t,i=100*i,h=100*h,[t,i,h]}function F(e){var t,i,n=e[0]/255,a=e[1]/255,o=e[2]/255,r=Math.min(n,a,o),s=Math.max(n,a,o),l=s-r,h=(s+r)/2;if(0===l)t=0,i=0;else{i=.5>h?l/(s+r):l/(2-s-r);var d=((s-n)/6+l/2)/l,m=((s-a)/6+l/2)/l,c=((s-o)/6+l/2)/l;n===s?t=c-m:a===s?t=1/3+d-c:o===s&&(t=2/3+m-d),0>t&&(t+=1),t>1&&(t-=1)}return t=360*t,i=100*i,h=100*h,[t,i,h]}var O,P=e("../tool/util"),D=["#ff9277"," #dddd00"," #ffc877"," #bbe3ff"," #d5ffbb","#bbbbff"," #ddb000"," #b0dd00"," #e2bbff"," #ffbbe3","#ff7777"," #ff9900"," #83dd00"," #77e3ff"," #778fff","#c877ff"," #ff77ab"," #ff6600"," #aa8800"," #77c7ff","#ad77ff"," #ff77ff"," #dd0083"," #777700"," #00aa00","#0088aa"," #8400dd"," #aa0088"," #dd0000"," #772e00"],N=D,H="rgba(255,255,0,0.5)",B=H,R=/^\s*((#[a-f\d]{6})|(#[a-f\d]{3})|rgba?\(\s*([\d\.]+%?\s*,\s*[\d\.]+%?\s*,\s*[\d\.]+%?(?:\s*,\s*[\d\.]+%?)?)\s*\)|hsba?\(\s*([\d\.]+(?:deg|\xb0|%)?\s*,\s*[\d\.]+%?\s*,\s*[\d\.]+%?(?:\s*,\s*[\d\.]+)?)%?\s*\)|hsla?\(\s*([\d\.]+(?:deg|\xb0|%)?\s*,\s*[\d\.]+%?\s*,\s*[\d\.]+%?(?:\s*,\s*[\d\.]+)?)%?\s*\))\s*$/i,G={aliceblue:"#f0f8ff",antiquewhite:"#faebd7",aqua:"#0ff",aquamarine:"#7fffd4",azure:"#f0ffff",beige:"#f5f5dc",bisque:"#ffe4c4",black:"#000",blanchedalmond:"#ffebcd",blue:"#00f",blueviolet:"#8a2be2",brown:"#a52a2a",burlywood:"#deb887",cadetblue:"#5f9ea0",chartreuse:"#7fff00",chocolate:"#d2691e",coral:"#ff7f50",cornflowerblue:"#6495ed",cornsilk:"#fff8dc",crimson:"#dc143c",cyan:"#0ff",darkblue:"#00008b",darkcyan:"#008b8b",darkgoldenrod:"#b8860b",darkgray:"#a9a9a9",darkgrey:"#a9a9a9",darkgreen:"#006400",darkkhaki:"#bdb76b",darkmagenta:"#8b008b",darkolivegreen:"#556b2f",darkorange:"#ff8c00",darkorchid:"#9932cc",darkred:"#8b0000",darksalmon:"#e9967a",darkseagreen:"#8fbc8f",darkslateblue:"#483d8b",darkslategray:"#2f4f4f",darkslategrey:"#2f4f4f",darkturquoise:"#00ced1",darkviolet:"#9400d3",deeppink:"#ff1493",deepskyblue:"#00bfff",dimgray:"#696969",dimgrey:"#696969",dodgerblue:"#1e90ff",firebrick:"#b22222",floralwhite:"#fffaf0",forestgreen:"#228b22",fuchsia:"#f0f",gainsboro:"#dcdcdc",ghostwhite:"#f8f8ff",gold:"#ffd700",goldenrod:"#daa520",gray:"#808080",grey:"#808080",green:"#008000",greenyellow:"#adff2f",honeydew:"#f0fff0",hotpink:"#ff69b4",indianred:"#cd5c5c",indigo:"#4b0082",ivory:"#fffff0",khaki:"#f0e68c",lavender:"#e6e6fa",lavenderblush:"#fff0f5",lawngreen:"#7cfc00",lemonchiffon:"#fffacd",lightblue:"#add8e6",lightcoral:"#f08080",lightcyan:"#e0ffff",lightgoldenrodyellow:"#fafad2",lightgray:"#d3d3d3",lightgrey:"#d3d3d3",lightgreen:"#90ee90",lightpink:"#ffb6c1",lightsalmon:"#ffa07a",lightseagreen:"#20b2aa",lightskyblue:"#87cefa",lightslategray:"#789",lightslategrey:"#789",lightsteelblue:"#b0c4de",lightyellow:"#ffffe0",lime:"#0f0",limegreen:"#32cd32",linen:"#faf0e6",magenta:"#f0f",maroon:"#800000",mediumaquamarine:"#66cdaa",mediumblue:"#0000cd",mediumorchid:"#ba55d3",mediumpurple:"#9370d8",mediumseagreen:"#3cb371",mediumslateblue:"#7b68ee",mediumspringgreen:"#00fa9a",mediumturquoise:"#48d1cc",mediumvioletred:"#c71585",midnightblue:"#191970",mintcream:"#f5fffa",mistyrose:"#ffe4e1",moccasin:"#ffe4b5",navajowhite:"#ffdead",navy:"#000080",oldlace:"#fdf5e6",olive:"#808000",olivedrab:"#6b8e23",orange:"#ffa500",orangered:"#ff4500",orchid:"#da70d6",palegoldenrod:"#eee8aa",palegreen:"#98fb98",paleturquoise:"#afeeee",palevioletred:"#d87093",papayawhip:"#ffefd5",peachpuff:"#ffdab9",peru:"#cd853f",pink:"#ffc0cb",plum:"#dda0dd",powderblue:"#b0e0e6",purple:"#800080",red:"#f00",rosybrown:"#bc8f8f",royalblue:"#4169e1",saddlebrown:"#8b4513",salmon:"#fa8072",sandybrown:"#f4a460",seagreen:"#2e8b57",seashell:"#fff5ee",sienna:"#a0522d",silver:"#c0c0c0",skyblue:"#87ceeb",slateblue:"#6a5acd",slategray:"#708090",slategrey:"#708090",snow:"#fffafa",springgreen:"#00ff7f",steelblue:"#4682b4",tan:"#d2b48c",teal:"#008080",thistle:"#d8bfd8",tomato:"#ff6347",turquoise:"#40e0d0",violet:"#ee82ee",wheat:"#f5deb3",white:"#fff",whitesmoke:"#f5f5f5",yellow:"#ff0",yellowgreen:"#9acd32"};return{customPalette:t,resetPalette:i,getColor:n,getHighlightColor:r,customHighlight:a,resetHighlight:o,getRadialGradient:s,getLinearGradient:l,getGradientColors:d,getStepColors:h,reverse:w,mix:X,lift:W,trim:L,random:I,toRGB:V,toRGBA:u,toHex:U,toHSL:x,toHSLA:_,toHSB:b,toHSBA:f,toHSV:g,toHSVA:y,toName:k,toColor:m,toArray:c,alpha:S,getData:K}}),define("echarts/component/timeline",["require","./base","zrender/shape/Rectangle","../util/shape/Icon","../util/shape/Chain","../config","zrender/tool/util","zrender/tool/area","zrender/tool/event","../component"],function(e){function t(e,t,i,a,o){n.call(this,e,t,i,a,o);var r=this;if(r._onclick=function(e){return r.__onclick(e)},r._ondrift=function(e,t){return r.__ondrift(this,e,t)},r._ondragend=function(){return r.__ondragend()},r._setCurrentOption=function(){var e=r.timelineOption;r.currentIndex%=e.data.length;var t=r.options[r.currentIndex]||{};r.myChart.setOption(t,e.notMerge),r.messageCenter.dispatch(s.EVENT.TIMELINE_CHANGED,null,{currentIndex:r.currentIndex,data:null!=e.data[r.currentIndex].name?e.data[r.currentIndex].name:e.data[r.currentIndex]},r.myChart)},r._onFrame=function(){r._setCurrentOption(),r._syncHandleShape(),r.timelineOption.autoPlay&&(r.playTicket=setTimeout(function(){return r.currentIndex+=1,!r.timelineOption.loop&&r.currentIndex>=r.timelineOption.data.length?(r.currentIndex=r.timelineOption.data.length-1,void r.stop()):void r._onFrame()},r.timelineOption.playInterval))},this.setTheme(!1),this.options=this.option.options,this.currentIndex=this.timelineOption.currentIndex%this.timelineOption.data.length,this.timelineOption.notMerge||0===this.currentIndex||(this.options[this.currentIndex]=l.merge(this.options[this.currentIndex],this.options[0])),this.timelineOption.show&&(this._buildShape(),this._syncHandleShape()),this._setCurrentOption(),this.timelineOption.autoPlay){var r=this;this.playTicket=setTimeout(function(){r.play()},this.ecTheme.animationDuration)}}function i(e,t){var i=2,n=t.x+i,a=t.y+i+2,r=t.width-i,s=t.height-i,l=t.symbol;if("last"===l)e.moveTo(n+r-2,a+s/3),e.lineTo(n+r-2,a),e.lineTo(n+2,a+s/2),e.lineTo(n+r-2,a+s),e.lineTo(n+r-2,a+s/3*2),e.moveTo(n,a),e.lineTo(n,a);else if("next"===l)e.moveTo(n+2,a+s/3),e.lineTo(n+2,a),e.lineTo(n+r-2,a+s/2),e.lineTo(n+2,a+s),e.lineTo(n+2,a+s/3*2),e.moveTo(n,a),e.lineTo(n,a);else if("play"===l)if("stop"===t.status)e.moveTo(n+2,a),e.lineTo(n+r-2,a+s/2),e.lineTo(n+2,a+s),e.lineTo(n+2,a);else{var h="both"===t.brushType?2:3;e.rect(n+2,a,h,s),e.rect(n+r-h-2,a,h,s)}else if(l.match("image")){var d="";d=l.replace(new RegExp("^image:\\/\\/"),""),l=o.prototype.iconLibrary.image,l(e,{x:n,y:a,width:r,height:s,image:d})}}var n=e("./base"),a=e("zrender/shape/Rectangle"),o=e("../util/shape/Icon"),r=e("../util/shape/Chain"),s=e("../config"),l=e("zrender/tool/util"),h=e("zrender/tool/area"),d=e("zrender/tool/event");return t.prototype={type:s.COMPONENT_TYPE_TIMELINE,_buildShape:function(){if(this._location=this._getLocation(),this._buildBackground(),this._buildControl(),this._chainPoint=this._getChainPoint(),this.timelineOption.label.show)for(var e=this._getInterval(),t=0,i=this._chainPoint.length;i>t;t+=e)this._chainPoint[t].showLabel=!0;this._buildChain(),this._buildHandle();for(var t=0,n=this.shapeList.length;n>t;t++)this.zr.addShape(this.shapeList[t])},_getLocation:function(){var e,t=this.timelineOption,i=this.reformCssArray(this.timelineOption.padding),n=this.zr.getWidth(),a=this.parsePercent(t.x,n),o=this.parsePercent(t.x2,n);null==t.width?(e=n-a-o,o=n-o):(e=this.parsePercent(t.width,n),o=a+e);var r,s,l=this.zr.getHeight(),h=this.parsePercent(t.height,l);return null!=t.y?(r=this.parsePercent(t.y,l),s=r+h):(s=l-this.parsePercent(t.y2,l),r=s-h),{x:a+i[3],y:r+i[0],x2:o-i[1],y2:s-i[2],width:e-i[1]-i[3],height:h-i[0]-i[2]}},_getReformedLabel:function(e){var t=this.timelineOption,i=null!=t.data[e].name?t.data[e].name:t.data[e],n=t.data[e].formatter||t.label.formatter;return n&&("function"==typeof n?i=n.call(this.myChart,i):"string"==typeof n&&(i=n.replace("{value}",i))),i},_getInterval:function(){var e=this._chainPoint,t=this.timelineOption,i=t.label.interval;if("auto"===i){var n=t.label.textStyle.fontSize,a=t.data,o=t.data.length;if(o>3){var r,s,l=!1;for(i=0;!l&&o>i;){i++,l=!0;for(var d=i;o>d;d+=i){if(r=e[d].x-e[d-i].x,0!==t.label.rotate)s=n;else if(a[d].textStyle)s=h.getTextWidth(e[d].name,e[d].textFont);else{var m=e[d].name+"",c=(m.match(/\w/g)||"").length,p=m.length-c;s=c*n*2/3+p*n}if(s>r){l=!1;break}}}}else i=1}else i=i-0+1;return i},_getChainPoint:function(){function e(e){return null!=h[e].name?h[e].name:h[e]+""}var t,i=this.timelineOption,n=i.symbol.toLowerCase(),a=i.symbolSize,o=i.label.rotate,r=i.label.textStyle,s=this.getFont(r),h=i.data,d=this._location.x,m=this._location.y+this._location.height/4*3,c=this._location.x2-this._location.x,p=h.length,u=[];if(p>1){var V=c/p;if(V=V>50?50:20>V?5:V,c-=2*V,"number"===i.type)for(var U=0;p>U;U++)u.push(d+V+c/(p-1)*U);else{u[0]=new Date(e(0).replace(/-/g,"/")),u[p-1]=new Date(e(p-1).replace(/-/g,"/"))-u[0];for(var U=1;p>U;U++)u[U]=d+V+c*(new Date(e(U).replace(/-/g,"/"))-u[0])/u[p-1];u[0]=d+V}}else u.push(d+c/2);for(var y,g,f,b,_,x=[],U=0;p>U;U++)d=u[U],y=h[U].symbol&&h[U].symbol.toLowerCase()||n,y.match("empty")?(y=y.replace("empty",""),f=!0):f=!1,y.match("star")&&(g=y.replace("star","")-0||5,y="star"),t=h[U].textStyle?l.merge(h[U].textStyle||{},r):r,b=t.align||"center",o?(b=o>0?"right":"left",_=[o*Math.PI/180,d,m-5]):_=!1,x.push({x:d,n:g,isEmpty:f,symbol:y,symbolSize:h[U].symbolSize||a,color:h[U].color,borderColor:h[U].borderColor,borderWidth:h[U].borderWidth,name:this._getReformedLabel(U),textColor:t.color,textAlign:b,textBaseline:t.baseline||"middle",textX:d,textY:m-(o?5:0),textFont:h[U].textStyle?this.getFont(t):s,rotation:_,showLabel:!1});return x},_buildBackground:function(){var e=this.timelineOption,t=this.reformCssArray(this.timelineOption.padding),i=this._location.width,n=this._location.height;(0!==e.borderWidth||"rgba(0,0,0,0)"!=e.backgroundColor.replace(/\s/g,""))&&this.shapeList.push(new a({zlevel:this._zlevelBase,hoverable:!1,style:{x:this._location.x-t[3],y:this._location.y-t[0],width:i+t[1]+t[3],height:n+t[0]+t[2],brushType:0===e.borderWidth?"fill":"both",color:e.backgroundColor,strokeColor:e.borderColor,lineWidth:e.borderWidth}}))},_buildControl:function(){var e=this,t=this.timelineOption,i=t.lineStyle,n=t.controlStyle;if("none"!==t.controlPosition){var a,r=15,s=5;"left"===t.controlPosition?(a=this._location.x,this._location.x+=3*(r+s)):(a=this._location.x2-(3*(r+s)-s),this._location.x2-=3*(r+s));var h=this._location.y,d={zlevel:this._zlevelBase+1,style:{iconType:"timelineControl",symbol:"last",x:a,y:h,width:r,height:r,brushType:"stroke",color:n.normal.color,strokeColor:n.normal.color,lineWidth:i.width},highlightStyle:{color:n.emphasis.color,strokeColor:n.emphasis.color,lineWidth:i.width+1},clickable:!0};this._ctrLastShape=new o(d),this._ctrLastShape.onclick=function(){e.last()},this.shapeList.push(this._ctrLastShape),a+=r+s,this._ctrPlayShape=new o(l.clone(d)),this._ctrPlayShape.style.brushType="fill",this._ctrPlayShape.style.symbol="play",this._ctrPlayShape.style.status=this.timelineOption.autoPlay?"playing":"stop",this._ctrPlayShape.style.x=a,this._ctrPlayShape.onclick=function(){"stop"===e._ctrPlayShape.style.status?e.play():e.stop()},this.shapeList.push(this._ctrPlayShape),a+=r+s,this._ctrNextShape=new o(l.clone(d)),this._ctrNextShape.style.symbol="next",this._ctrNextShape.style.x=a,this._ctrNextShape.onclick=function(){e.next()},this.shapeList.push(this._ctrNextShape)}},_buildChain:function(){var e=this.timelineOption,t=e.lineStyle;this._timelineShae={zlevel:this._zlevelBase,style:{x:this._location.x,y:this.subPixelOptimize(this._location.y,t.width),width:this._location.x2-this._location.x,height:this._location.height,chainPoint:this._chainPoint,brushType:"both",strokeColor:t.color,lineWidth:t.width,lineType:t.type},hoverable:!1,clickable:!0,onclick:this._onclick},this._timelineShae=new r(this._timelineShae),this.shapeList.push(this._timelineShae)},_buildHandle:function(){var e=this._chainPoint[this.currentIndex],t=e.symbolSize+1;t=5>t?5:t,this._handleShape={zlevel:this._zlevelBase+1,hoverable:!1,draggable:!0,style:{iconType:"diamond",n:e.n,x:e.x-t,y:this._location.y+this._location.height/4-t,width:2*t,height:2*t,brushType:"both",textPosition:"specific",textX:e.x,textY:this._location.y-this._location.height/4,textAlign:"center",textBaseline:"middle"},highlightStyle:{},ondrift:this._ondrift,ondragend:this._ondragend},this._handleShape=new o(this._handleShape),this.shapeList.push(this._handleShape)},_syncHandleShape:function(){if(this.timelineOption.show){var e=this.timelineOption,t=e.checkpointStyle,i=this._chainPoint[this.currentIndex];this._handleShape.style.text=t.label.show?i.name:"",this._handleShape.style.textFont=i.textFont,this._handleShape.style.n=i.n,"auto"===t.symbol?this._handleShape.style.iconType="none"!=i.symbol?i.symbol:"diamond":(this._handleShape.style.iconType=t.symbol,t.symbol.match("star")&&(this._handleShape.style.n=t.symbol.replace("star","")-0||5,this._handleShape.style.iconType="star"));var n;"auto"===t.symbolSize?(n=i.symbolSize+2,n=5>n?5:n):n=t.symbolSize-0,this._handleShape.style.color="auto"===t.color?i.color?i.color:e.controlStyle.emphasis.color:t.color,this._handleShape.style.textColor="auto"===t.label.textStyle.color?this._handleShape.style.color:t.label.textStyle.color,this._handleShape.highlightStyle.strokeColor=this._handleShape.style.strokeColor="auto"===t.borderColor?i.borderColor?i.borderColor:"#fff":t.borderColor,this._handleShape.style.lineWidth="auto"===t.borderWidth?i.borderWidth?i.borderWidth:0:t.borderWidth-0,this._handleShape.highlightStyle.lineWidth=this._handleShape.style.lineWidth+1,this.zr.animate(this._handleShape.id,"style").when(500,{x:i.x-n,textX:i.x,y:this._location.y+this._location.height/4-n,width:2*n,height:2*n}).start("ExponentialOut") +}},_findChainIndex:function(e){var t=this._chainPoint,i=t.length;if(e<=t[0].x)return 0;if(e>=t[i-1].x)return i-1;for(var n=0;i-1>n;n++)if(e>=t[n].x&&e<=t[n+1].x)return Math.abs(e-t[n].x)=n[a-1].x-n[a-1].symbolSize?(e.style.x=n[a-1].x-n[a-1].symbolSize,i=a-1):(e.style.x+=t,i=this._findChainIndex(e.style.x));var o=n[i],r=o.symbolSize+2;if(e.style.iconType=o.symbol,e.style.n=o.n,e.style.textX=e.style.x+r/2,e.style.y=this._location.y+this._location.height/4-r,e.style.width=2*r,e.style.height=2*r,e.style.text=o.name,i===this.currentIndex)return!0;if(this.currentIndex=i,this.timelineOption.realtime){clearTimeout(this.playTicket);var s=this;this.playTicket=setTimeout(function(){s._setCurrentOption()},200)}return!0},__ondragend:function(){this.isDragend=!0},ondragend:function(e,t){this.isDragend&&e.target&&(!this.timelineOption.realtime&&this._setCurrentOption(),t.dragOut=!0,t.dragIn=!0,t.needRefresh=!1,this.isDragend=!1,this._syncHandleShape())},last:function(){return this.timelineOption.autoPlay&&this.stop(),this.currentIndex-=1,this.currentIndex<0&&(this.currentIndex=this.timelineOption.data.length-1),this._onFrame(),this.currentIndex},next:function(){return this.timelineOption.autoPlay&&this.stop(),this.currentIndex+=1,this.currentIndex>=this.timelineOption.data.length&&(this.currentIndex=0),this._onFrame(),this.currentIndex},play:function(e,t){return this._ctrPlayShape&&"playing"!=this._ctrPlayShape.style.status&&(this._ctrPlayShape.style.status="playing",this.zr.modShape(this._ctrPlayShape.id),this.zr.refresh()),this.timelineOption.autoPlay=null!=t?t:!0,this.timelineOption.autoPlay||clearTimeout(this.playTicket),this.currentIndex=null!=e?e:this.currentIndex+1,this.currentIndex>=this.timelineOption.data.length&&(this.currentIndex=0),this._onFrame(),this.currentIndex},stop:function(){return this._ctrPlayShape&&"stop"!=this._ctrPlayShape.style.status&&(this._ctrPlayShape.style.status="stop",this.zr.modShape(this._ctrPlayShape.id),this.zr.refresh()),this.timelineOption.autoPlay=!1,clearTimeout(this.playTicket),this.currentIndex},resize:function(){this.timelineOption.show&&(this.clear(),this._buildShape(),this._syncHandleShape())},setTheme:function(e){this.timelineOption=this.reformOption(l.clone(this.option.timeline)),this.timelineOption.label.textStyle=l.merge(this.timelineOption.label.textStyle||{},this.ecTheme.textStyle),this.timelineOption.checkpointStyle.label.textStyle=l.merge(this.timelineOption.checkpointStyle.label.textStyle||{},this.ecTheme.textStyle),this.myChart.canvasSupported||(this.timelineOption.realtime=!1),this.timelineOption.show&&e&&(this.clear(),this._buildShape(),this._syncHandleShape())},onbeforDispose:function(){clearTimeout(this.playTicket)}},o.prototype.iconLibrary.timelineControl=i,l.inherits(t,n),e("../component").define("timeline",t),t}),define("zrender/shape/Image",["require","./Base","../tool/util"],function(e){var t=e("./Base"),i=function(e){t.call(this,e)};return i.prototype={type:"image",brush:function(e,t,i){var n=this.style||{};t&&(n=this.getHighlightStyle(n,this.highlightStyle||{}));var a=n.image,o=this;if(this._imageCache||(this._imageCache={}),"string"==typeof a){var r=a;this._imageCache[r]?a=this._imageCache[r]:(a=new Image,a.onload=function(){a.onload=null,o.modSelf(),i()},a.src=r,this._imageCache[r]=a)}if(a){if("IMG"==a.nodeName.toUpperCase())if(window.ActiveXObject){if("complete"!=a.readyState)return}else if(!a.complete)return;var s=n.width||a.width,l=n.height||a.height,h=n.x,d=n.y;if(!a.width||!a.height)return;if(e.save(),this.doClip(e),this.setContext(e,n),this.setTransform(e),n.sWidth&&n.sHeight){var m=n.sx||0,c=n.sy||0;e.drawImage(a,m,c,n.sWidth,n.sHeight,h,d,s,l)}else if(n.sx&&n.sy){var m=n.sx,c=n.sy,p=s-m,u=l-c;e.drawImage(a,m,c,p,u,h,d,s,l)}else e.drawImage(a,h,d,s,l);n.width||(n.width=s),n.height||(n.height=l),this.style.width||(this.style.width=s),this.style.height||(this.style.height=l),this.drawText(e,n,this.style),e.restore()}},getRect:function(e){return{x:e.x,y:e.y,width:e.width,height:e.height}},clearCache:function(){this._imageCache={}}},e("../tool/util").inherits(i,t),i}),define("zrender/loadingEffect/Bar",["require","./Base","../tool/util","../tool/color","../shape/Rectangle"],function(e){function t(e){i.call(this,e)}var i=e("./Base"),n=e("../tool/util"),a=e("../tool/color"),o=e("../shape/Rectangle");return n.inherits(t,i),t.prototype._start=function(e,t){var i=n.merge(this.options,{textStyle:{color:"#888"},backgroundColor:"rgba(250, 250, 250, 0.8)",effectOption:{x:0,y:this.canvasHeight/2-30,width:this.canvasWidth,height:5,brushType:"fill",timeInterval:100}}),r=this.createTextShape(i.textStyle),s=this.createBackgroundShape(i.backgroundColor),l=i.effectOption,h=new o({highlightStyle:n.clone(l)});return h.highlightStyle.color=l.color||a.getLinearGradient(l.x,l.y,l.x+l.width,l.y+l.height,[[0,"#ff6400"],[.5,"#ffe100"],[1,"#b1ff00"]]),null!=i.progress?(e(s),h.highlightStyle.width=this.adjust(i.progress,[0,1])*i.effectOption.width,e(h),e(r),void t()):(h.highlightStyle.width=0,setInterval(function(){e(s),h.highlightStyle.widthV;V++){var U="random"==l.color?a.alpha(a.random(),.3):l.color;c[V]=new o({highlightStyle:{x:Math.ceil(Math.random()*p),y:Math.ceil(Math.random()*u),r:Math.ceil(40*Math.random()),brushType:d,color:U,strokeColor:U,lineWidth:m},animationY:Math.ceil(20*Math.random())})}return setInterval(function(){e(s);for(var i=0;h>i;i++){var n=c[i].highlightStyle;n.y-c[i].animationY+n.r<=0&&(c[i].highlightStyle.y=u+n.r,c[i].highlightStyle.x=Math.ceil(Math.random()*p)),c[i].highlightStyle.y-=c[i].animationY,e(c[i])}e(r),t()},l.timeInterval)},t}),define("zrender/loadingEffect/DynamicLine",["require","./Base","../tool/util","../tool/color","../shape/Line"],function(e){function t(e){i.call(this,e)}var i=e("./Base"),n=e("../tool/util"),a=e("../tool/color"),o=e("../shape/Line");return n.inherits(t,i),t.prototype._start=function(e,t){for(var i=n.merge(this.options,{textStyle:{color:"#fff"},backgroundColor:"rgba(0, 0, 0, 0.8)",effectOption:{n:30,lineWidth:1,color:"random",timeInterval:100}}),r=this.createTextShape(i.textStyle),s=this.createBackgroundShape(i.backgroundColor),l=i.effectOption,h=l.n,d=l.lineWidth,m=[],c=this.canvasWidth,p=this.canvasHeight,u=0;h>u;u++){var V=-Math.ceil(1e3*Math.random()),U=Math.ceil(400*Math.random()),y=Math.ceil(Math.random()*p),g="random"==l.color?a.random():l.color;m[u]=new o({highlightStyle:{xStart:V,yStart:y,xEnd:V+U,yEnd:y,strokeColor:g,lineWidth:d},animationX:Math.ceil(100*Math.random()),len:U})}return setInterval(function(){e(s);for(var i=0;h>i;i++){var n=m[i].highlightStyle;n.xStart>=c&&(m[i].len=Math.ceil(400*Math.random()),n.xStart=-400,n.xEnd=-400+m[i].len,n.yStart=Math.ceil(Math.random()*p),n.yEnd=n.yStart),n.xStart+=m[i].animationX,n.xEnd+=m[i].animationX,e(m[i])}e(r),t()},l.timeInterval)},t}),define("zrender/loadingEffect/Ring",["require","./Base","../tool/util","../tool/color","../shape/Ring","../shape/Sector"],function(e){function t(e){i.call(this,e)}var i=e("./Base"),n=e("../tool/util"),a=e("../tool/color"),o=e("../shape/Ring"),r=e("../shape/Sector");return n.inherits(t,i),t.prototype._start=function(e,t){var i=n.merge(this.options,{textStyle:{color:"#07a"},backgroundColor:"rgba(250, 250, 250, 0.8)",effect:{x:this.canvasWidth/2,y:this.canvasHeight/2,r0:60,r:100,color:"#bbdcff",brushType:"fill",textPosition:"inside",textFont:"normal 30px verdana",textColor:"rgba(30, 144, 255, 0.6)",timeInterval:100}}),s=i.effect,l=i.textStyle;null==l.x&&(l.x=s.x),null==l.y&&(l.y=s.y+(s.r0+s.r)/2-5);for(var h=this.createTextShape(i.textStyle),d=this.createBackgroundShape(i.backgroundColor),m=s.x,c=s.y,p=s.r0+6,u=s.r-6,V=s.color,U=a.lift(V,.1),y=new o({highlightStyle:n.clone(s)}),g=[],f=a.getGradientColors(["#ff6400","#ffe100","#97ff00"],25),b=15,_=240,x=0;16>x;x++)g.push(new r({highlightStyle:{x:m,y:c,r0:p,r:u,startAngle:_-b,endAngle:_,brushType:"fill",color:U},_color:a.getLinearGradient(m+p*Math.cos(_,!0),c-p*Math.sin(_,!0),m+p*Math.cos(_-b,!0),c-p*Math.sin(_-b,!0),[[0,f[2*x]],[1,f[2*x+1]]])})),_-=b;_=360;for(var x=0;4>x;x++)g.push(new r({highlightStyle:{x:m,y:c,r0:p,r:u,startAngle:_-b,endAngle:_,brushType:"fill",color:U},_color:a.getLinearGradient(m+p*Math.cos(_,!0),c-p*Math.sin(_,!0),m+p*Math.cos(_-b,!0),c-p*Math.sin(_-b,!0),[[0,f[2*x+32]],[1,f[2*x+33]]])})),_-=b;var k=0;if(null!=i.progress){e(d),k=100*this.adjust(i.progress,[0,1]).toFixed(2)/5,y.highlightStyle.text=5*k+"%",e(y);for(var x=0;20>x;x++)g[x].highlightStyle.color=k>x?g[x]._color:U,e(g[x]);return e(h),void t()}return setInterval(function(){e(d),k+=k>=20?-20:1,e(y);for(var i=0;20>i;i++)g[i].highlightStyle.color=k>i?g[i]._color:U,e(g[i]);e(h),t()},s.timeInterval)},t}),define("zrender/loadingEffect/Spin",["require","./Base","../tool/util","../tool/color","../tool/area","../shape/Sector"],function(e){function t(e){i.call(this,e)}var i=e("./Base"),n=e("../tool/util"),a=e("../tool/color"),o=e("../tool/area"),r=e("../shape/Sector");return n.inherits(t,i),t.prototype._start=function(e,t){var i=n.merge(this.options,{textStyle:{color:"#fff",textAlign:"start"},backgroundColor:"rgba(0, 0, 0, 0.8)"}),s=this.createTextShape(i.textStyle),l=10,h=o.getTextWidth(s.highlightStyle.text,s.highlightStyle.textFont),d=o.getTextHeight(s.highlightStyle.text,s.highlightStyle.textFont),m=n.merge(this.options.effect||{},{r0:9,r:15,n:18,color:"#fff",timeInterval:100}),c=this.getLocation(this.options.textStyle,h+l+2*m.r,Math.max(2*m.r,d));m.x=c.x+m.r,m.y=s.highlightStyle.y=c.y+c.height/2,s.highlightStyle.x=m.x+m.r+l;for(var p=this.createBackgroundShape(i.backgroundColor),u=m.n,V=m.x,U=m.y,y=m.r0,g=m.r,f=m.color,b=[],_=Math.round(180/u),x=0;u>x;x++)b[x]=new r({highlightStyle:{x:V,y:U,r0:y,r:g,startAngle:_*x*2,endAngle:_*x*2+_,color:a.alpha(f,(x+1)/u),brushType:"fill"}});var k=[0,V,U];return setInterval(function(){e(p),k[0]-=.3;for(var i=0;u>i;i++)b[i].rotation=k,e(b[i]);e(s),t()},m.timeInterval)},t}),define("zrender/loadingEffect/Whirling",["require","./Base","../tool/util","../tool/area","../shape/Ring","../shape/Droplet","../shape/Circle"],function(e){function t(e){i.call(this,e)}var i=e("./Base"),n=e("../tool/util"),a=e("../tool/area"),o=e("../shape/Ring"),r=e("../shape/Droplet"),s=e("../shape/Circle");return n.inherits(t,i),t.prototype._start=function(e,t){var i=n.merge(this.options,{textStyle:{color:"#888",textAlign:"start"},backgroundColor:"rgba(250, 250, 250, 0.8)"}),l=this.createTextShape(i.textStyle),h=10,d=a.getTextWidth(l.highlightStyle.text,l.highlightStyle.textFont),m=a.getTextHeight(l.highlightStyle.text,l.highlightStyle.textFont),c=n.merge(this.options.effect||{},{r:18,colorIn:"#fff",colorOut:"#555",colorWhirl:"#6cf",timeInterval:50}),p=this.getLocation(this.options.textStyle,d+h+2*c.r,Math.max(2*c.r,m));c.x=p.x+c.r,c.y=l.highlightStyle.y=p.y+p.height/2,l.highlightStyle.x=c.x+c.r+h;var u=this.createBackgroundShape(i.backgroundColor),V=new r({highlightStyle:{a:Math.round(c.r/2),b:Math.round(c.r-c.r/6),brushType:"fill",color:c.colorWhirl}}),U=new s({highlightStyle:{r:Math.round(c.r/6),brushType:"fill",color:c.colorIn}}),y=new o({highlightStyle:{r0:Math.round(c.r-c.r/3),r:c.r,brushType:"fill",color:c.colorOut}}),g=[0,c.x,c.y];return V.highlightStyle.x=U.highlightStyle.x=y.highlightStyle.x=g[1],V.highlightStyle.y=U.highlightStyle.y=y.highlightStyle.y=g[2],setInterval(function(){e(u),e(y),g[0]-=.3,V.rotation=g,e(V),e(U),e(l),t()},c.timeInterval)},t}),define("echarts/theme/default",[],function(){var e={};return e}),define("zrender/dep/excanvas",["require"],function(){return document.createElement("canvas").getContext?G_vmlCanvasManager=!1:!function(){function e(){return this.context_||(this.context_=new b(this))}function t(e,t){var i=O.call(arguments,2);return function(){return e.apply(t,i.concat(O.call(arguments)))}}function i(e){return String(e).replace(/&/g,"&").replace(/"/g,""")}function n(e,t,i){e.namespaces[t]||e.namespaces.add(t,i,"#default#VML")}function a(e){if(n(e,"g_vml_","urn:schemas-microsoft-com:vml"),n(e,"g_o_","urn:schemas-microsoft-com:office:office"),!e.styleSheets.ex_canvas_){var t=e.createStyleSheet();t.owningElement.id="ex_canvas_",t.cssText="canvas{display:inline-block;overflow:hidden;text-align:left;width:300px;height:150px}"}}function o(e){var t=e.srcElement;switch(e.propertyName){case"width":t.getContext().clearRect(),t.style.width=t.attributes.width.nodeValue+"px",t.firstChild.style.width=t.clientWidth+"px";break;case"height":t.getContext().clearRect(),t.style.height=t.attributes.height.nodeValue+"px",t.firstChild.style.height=t.clientHeight+"px"}}function r(e){var t=e.srcElement;t.firstChild&&(t.firstChild.style.width=t.clientWidth+"px",t.firstChild.style.height=t.clientHeight+"px")}function s(){return[[1,0,0],[0,1,0],[0,0,1]]}function l(e,t){for(var i=s(),n=0;3>n;n++)for(var a=0;3>a;a++){for(var o=0,r=0;3>r;r++)o+=e[n][r]*t[r][a];i[n][a]=o}return i}function h(e,t){t.fillStyle=e.fillStyle,t.lineCap=e.lineCap,t.lineJoin=e.lineJoin,t.lineWidth=e.lineWidth,t.miterLimit=e.miterLimit,t.shadowBlur=e.shadowBlur,t.shadowColor=e.shadowColor,t.shadowOffsetX=e.shadowOffsetX,t.shadowOffsetY=e.shadowOffsetY,t.strokeStyle=e.strokeStyle,t.globalAlpha=e.globalAlpha,t.font=e.font,t.textAlign=e.textAlign,t.textBaseline=e.textBaseline,t.scaleX_=e.scaleX_,t.scaleY_=e.scaleY_,t.lineScale_=e.lineScale_}function d(e){var t=e.indexOf("(",3),i=e.indexOf(")",t+1),n=e.substring(t+1,i).split(",");return(4!=n.length||"a"!=e.charAt(3))&&(n[3]=1),n}function m(e){return parseFloat(e)/100}function c(e,t,i){return Math.min(i,Math.max(t,e))}function p(e){var t,i,n,a,o,r;if(a=parseFloat(e[0])/360%360,0>a&&a++,o=c(m(e[1]),0,1),r=c(m(e[2]),0,1),0==o)t=i=n=r;else{var s=.5>r?r*(1+o):r+o-r*o,l=2*r-s;t=u(l,s,a+1/3),i=u(l,s,a),n=u(l,s,a-1/3)}return"#"+D[Math.floor(255*t)]+D[Math.floor(255*i)]+D[Math.floor(255*n)]}function u(e,t,i){return 0>i&&i++,i>1&&i--,1>6*i?e+6*(t-e)*i:1>2*i?t:2>3*i?e+(t-e)*(2/3-i)*6:e}function V(e){if(e in R)return R[e];var t,i=1;if(e=String(e),"#"==e.charAt(0))t=e;else if(/^rgb/.test(e)){for(var n,a=d(e),t="#",o=0;3>o;o++)n=-1!=a[o].indexOf("%")?Math.floor(255*m(a[o])):+a[o],t+=D[c(n,0,255)];i=+a[3]}else if(/^hsl/.test(e)){var a=d(e);t=p(a),i=a[3]}else t=B[e]||e;return R[e]={color:t,alpha:i}}function U(e){if(Y[e])return Y[e];var t,i=document.createElement("div"),n=i.style;try{n.font=e,t=n.fontFamily.split(",")[0]}catch(a){}return Y[e]={style:n.fontStyle||G.style,variant:n.fontVariant||G.variant,weight:n.fontWeight||G.weight,size:n.fontSize||G.size,family:t||G.family}}function y(e,t){var i={};for(var n in e)i[n]=e[n];var a=parseFloat(t.currentStyle.fontSize),o=parseFloat(e.size);return i.size="number"==typeof e.size?e.size:-1!=e.size.indexOf("px")?o:-1!=e.size.indexOf("em")?a*o:-1!=e.size.indexOf("%")?a/100*o:-1!=e.size.indexOf("pt")?o/.75:a,i}function g(e){return e.style+" "+e.variant+" "+e.weight+" "+e.size+"px '"+e.family+"'"}function f(e){return Q[e]||"square"}function b(e){this.m_=s(),this.mStack_=[],this.aStack_=[],this.currentPath_=[],this.strokeStyle="#000",this.fillStyle="#000",this.lineWidth=1,this.lineJoin="miter",this.lineCap="butt",this.miterLimit=1*J,this.globalAlpha=1,this.font="12px 微软雅黑",this.textAlign="left",this.textBaseline="alphabetic",this.canvas=e;var t="width:"+e.clientWidth+"px;height:"+e.clientHeight+"px;overflow:hidden;position:absolute",i=e.ownerDocument.createElement("div");i.style.cssText=t,e.appendChild(i);var n=i.cloneNode(!1);n.style.backgroundColor="#fff",n.style.filter="alpha(opacity=0)",e.appendChild(n),this.element_=i,this.scaleX_=1,this.scaleY_=1,this.lineScale_=1}function _(e,t,i,n){e.currentPath_.push({type:"bezierCurveTo",cp1x:t.x,cp1y:t.y,cp2x:i.x,cp2y:i.y,x:n.x,y:n.y}),e.currentX_=n.x,e.currentY_=n.y}function x(e,t){var i=V(e.strokeStyle),n=i.color,a=i.alpha*e.globalAlpha,o=e.lineScale_*e.lineWidth;1>o&&(a*=o),t.push("')}function k(e,t,i,n){var a=e.fillStyle,o=e.scaleX_,r=e.scaleY_,s=n.x-i.x,l=n.y-i.y;if(a instanceof w){var h=0,d={x:0,y:0},m=0,c=1;if("gradient"==a.type_){var p=a.x0_/o,u=a.y0_/r,U=a.x1_/o,y=a.y1_/r,g=L(e,p,u),f=L(e,U,y),b=f.x-g.x,_=f.y-g.y;h=180*Math.atan2(b,_)/Math.PI,0>h&&(h+=360),1e-6>h&&(h=0)}else{var g=L(e,a.x0_,a.y0_);d={x:(g.x-i.x)/s,y:(g.y-i.y)/l},s/=o*J,l/=r*J;var x=T.max(s,l);m=2*a.r0_/x,c=2*a.r1_/x-m}var k=a.colors_;k.sort(function(e,t){return e.offset-t.offset});for(var v=k.length,W=k[0].color,I=k[v-1].color,K=k[0].alpha*e.globalAlpha,S=k[v-1].alpha*e.globalAlpha,C=[],E=0;v>E;E++){var A=k[E];C.push(A.offset*c+m+" "+A.color)}t.push('')}else if(a instanceof X){if(s&&l){var z=-i.x,M=-i.y;t.push("')}}else{var F=V(e.fillStyle),O=F.color,P=F.alpha*e.globalAlpha;t.push('')}}function L(e,t,i){var n=e.m_;return{x:J*(t*n[0][0]+i*n[1][0]+n[2][0])-F,y:J*(t*n[0][1]+i*n[1][1]+n[2][1])-F}}function v(e){return isFinite(e[0][0])&&isFinite(e[0][1])&&isFinite(e[1][0])&&isFinite(e[1][1])&&isFinite(e[2][0])&&isFinite(e[2][1])}function W(e,t,i){if(v(t)&&(e.m_=t,e.scaleX_=Math.sqrt(t[0][0]*t[0][0]+t[0][1]*t[0][1]),e.scaleY_=Math.sqrt(t[1][0]*t[1][0]+t[1][1]*t[1][1]),i)){var n=t[0][0]*t[1][1]-t[0][1]*t[1][0];e.lineScale_=M(z(n))}}function w(e){this.type_=e,this.x0_=0,this.y0_=0,this.r0_=0,this.x1_=0,this.y1_=0,this.r1_=0,this.colors_=[]}function X(e,t){switch(K(e),t){case"repeat":case null:case"":this.repetition_="repeat";break;case"repeat-x":case"repeat-y":case"no-repeat":this.repetition_=t;break;default:I("SYNTAX_ERR")}this.src_=e.src,this.width_=e.width,this.height_=e.height}function I(e){throw new S(e)}function K(e){e&&1==e.nodeType&&"IMG"==e.tagName||I("TYPE_MISMATCH_ERR"),"complete"!=e.readyState&&I("INVALID_STATE_ERR")}function S(e){this.code=this[e],this.message=e+": DOM Exception "+this.code}var T=Math,C=T.round,E=T.sin,A=T.cos,z=T.abs,M=T.sqrt,J=10,F=J/2,O=(+navigator.userAgent.match(/MSIE ([\d.]+)?/)[1],Array.prototype.slice);a(document);var P={init:function(e){var i=e||document;i.createElement("canvas"),i.attachEvent("onreadystatechange",t(this.init_,this,i))},init_:function(e){for(var t=e.getElementsByTagName("canvas"),i=0;iN;N++)for(var H=0;16>H;H++)D[16*N+H]=N.toString(16)+H.toString(16);var B={aliceblue:"#F0F8FF",antiquewhite:"#FAEBD7",aquamarine:"#7FFFD4",azure:"#F0FFFF",beige:"#F5F5DC",bisque:"#FFE4C4",black:"#000000",blanchedalmond:"#FFEBCD",blueviolet:"#8A2BE2",brown:"#A52A2A",burlywood:"#DEB887",cadetblue:"#5F9EA0",chartreuse:"#7FFF00",chocolate:"#D2691E",coral:"#FF7F50",cornflowerblue:"#6495ED",cornsilk:"#FFF8DC",crimson:"#DC143C",cyan:"#00FFFF",darkblue:"#00008B",darkcyan:"#008B8B",darkgoldenrod:"#B8860B",darkgray:"#A9A9A9",darkgreen:"#006400",darkgrey:"#A9A9A9",darkkhaki:"#BDB76B",darkmagenta:"#8B008B",darkolivegreen:"#556B2F",darkorange:"#FF8C00",darkorchid:"#9932CC",darkred:"#8B0000",darksalmon:"#E9967A",darkseagreen:"#8FBC8F",darkslateblue:"#483D8B",darkslategray:"#2F4F4F",darkslategrey:"#2F4F4F",darkturquoise:"#00CED1",darkviolet:"#9400D3",deeppink:"#FF1493",deepskyblue:"#00BFFF",dimgray:"#696969",dimgrey:"#696969",dodgerblue:"#1E90FF",firebrick:"#B22222",floralwhite:"#FFFAF0",forestgreen:"#228B22",gainsboro:"#DCDCDC",ghostwhite:"#F8F8FF",gold:"#FFD700",goldenrod:"#DAA520",grey:"#808080",greenyellow:"#ADFF2F",honeydew:"#F0FFF0",hotpink:"#FF69B4",indianred:"#CD5C5C",indigo:"#4B0082",ivory:"#FFFFF0",khaki:"#F0E68C",lavender:"#E6E6FA",lavenderblush:"#FFF0F5",lawngreen:"#7CFC00",lemonchiffon:"#FFFACD",lightblue:"#ADD8E6",lightcoral:"#F08080",lightcyan:"#E0FFFF",lightgoldenrodyellow:"#FAFAD2",lightgreen:"#90EE90",lightgrey:"#D3D3D3",lightpink:"#FFB6C1",lightsalmon:"#FFA07A",lightseagreen:"#20B2AA",lightskyblue:"#87CEFA",lightslategray:"#778899",lightslategrey:"#778899",lightsteelblue:"#B0C4DE",lightyellow:"#FFFFE0",limegreen:"#32CD32",linen:"#FAF0E6",magenta:"#FF00FF",mediumaquamarine:"#66CDAA",mediumblue:"#0000CD",mediumorchid:"#BA55D3",mediumpurple:"#9370DB",mediumseagreen:"#3CB371",mediumslateblue:"#7B68EE",mediumspringgreen:"#00FA9A",mediumturquoise:"#48D1CC",mediumvioletred:"#C71585",midnightblue:"#191970",mintcream:"#F5FFFA",mistyrose:"#FFE4E1",moccasin:"#FFE4B5",navajowhite:"#FFDEAD",oldlace:"#FDF5E6",olivedrab:"#6B8E23",orange:"#FFA500",orangered:"#FF4500",orchid:"#DA70D6",palegoldenrod:"#EEE8AA",palegreen:"#98FB98",paleturquoise:"#AFEEEE",palevioletred:"#DB7093",papayawhip:"#FFEFD5",peachpuff:"#FFDAB9",peru:"#CD853F",pink:"#FFC0CB",plum:"#DDA0DD",powderblue:"#B0E0E6",rosybrown:"#BC8F8F",royalblue:"#4169E1",saddlebrown:"#8B4513",salmon:"#FA8072",sandybrown:"#F4A460",seagreen:"#2E8B57",seashell:"#FFF5EE",sienna:"#A0522D",skyblue:"#87CEEB",slateblue:"#6A5ACD",slategray:"#708090",slategrey:"#708090",snow:"#FFFAFA",springgreen:"#00FF7F",steelblue:"#4682B4",tan:"#D2B48C",thistle:"#D8BFD8",tomato:"#FF6347",turquoise:"#40E0D0",violet:"#EE82EE",wheat:"#F5DEB3",whitesmoke:"#F5F5F5",yellowgreen:"#9ACD32"},R={},G={style:"normal",variant:"normal",weight:"normal",size:12,family:"微软雅黑"},Y={},Q={butt:"flat",round:"round"},q=b.prototype;q.clearRect=function(){this.textMeasureEl_&&(this.textMeasureEl_.removeNode(!0),this.textMeasureEl_=null),this.element_.innerHTML=""},q.beginPath=function(){this.currentPath_=[]},q.moveTo=function(e,t){var i=L(this,e,t);this.currentPath_.push({type:"moveTo",x:i.x,y:i.y}),this.currentX_=i.x,this.currentY_=i.y},q.lineTo=function(e,t){var i=L(this,e,t);this.currentPath_.push({type:"lineTo",x:i.x,y:i.y}),this.currentX_=i.x,this.currentY_=i.y},q.bezierCurveTo=function(e,t,i,n,a,o){var r=L(this,a,o),s=L(this,e,t),l=L(this,i,n);_(this,s,l,r)},q.quadraticCurveTo=function(e,t,i,n){var a=L(this,e,t),o=L(this,i,n),r={x:this.currentX_+2/3*(a.x-this.currentX_),y:this.currentY_+2/3*(a.y-this.currentY_)},s={x:r.x+(o.x-this.currentX_)/3,y:r.y+(o.y-this.currentY_)/3};_(this,r,s,o)},q.arc=function(e,t,i,n,a,o){i*=J;var r=o?"at":"wa",s=e+A(n)*i-F,l=t+E(n)*i-F,h=e+A(a)*i-F,d=t+E(a)*i-F;s!=h||o||(s+=.125);var m=L(this,e,t),c=L(this,s,l),p=L(this,h,d);this.currentPath_.push({type:r,x:m.x,y:m.y,radius:i,xStart:c.x,yStart:c.y,xEnd:p.x,yEnd:p.y})},q.rect=function(e,t,i,n){this.moveTo(e,t),this.lineTo(e+i,t),this.lineTo(e+i,t+n),this.lineTo(e,t+n),this.closePath()},q.strokeRect=function(e,t,i,n){var a=this.currentPath_;this.beginPath(),this.moveTo(e,t),this.lineTo(e+i,t),this.lineTo(e+i,t+n),this.lineTo(e,t+n),this.closePath(),this.stroke(),this.currentPath_=a},q.fillRect=function(e,t,i,n){var a=this.currentPath_;this.beginPath(),this.moveTo(e,t),this.lineTo(e+i,t),this.lineTo(e+i,t+n),this.lineTo(e,t+n),this.closePath(),this.fill(),this.currentPath_=a},q.createLinearGradient=function(e,t,i,n){var a=new w("gradient");return a.x0_=e,a.y0_=t,a.x1_=i,a.y1_=n,a},q.createRadialGradient=function(e,t,i,n,a,o){var r=new w("gradientradial");return r.x0_=e,r.y0_=t,r.r0_=i,r.x1_=n,r.y1_=a,r.r1_=o,r},q.drawImage=function(e){var t,i,n,a,o,r,s,l,h=e.runtimeStyle.width,d=e.runtimeStyle.height;e.runtimeStyle.width="auto",e.runtimeStyle.height="auto";var m=e.width,c=e.height;if(e.runtimeStyle.width=h,e.runtimeStyle.height=d,3==arguments.length)t=arguments[1],i=arguments[2],o=r=0,s=n=m,l=a=c;else if(5==arguments.length)t=arguments[1],i=arguments[2],n=arguments[3],a=arguments[4],o=r=0,s=m,l=c;else{if(9!=arguments.length)throw Error("Invalid number of arguments");o=arguments[1],r=arguments[2],s=arguments[3],l=arguments[4],t=arguments[5],i=arguments[6],n=arguments[7],a=arguments[8]}var p=L(this,t,i),u=[],V=10,U=10,y=f=1;if(u.push(" '),(o||r)&&u.push('
'),u.push('
'),(o||r)&&u.push("
"),u.push("
"),this.element_.insertAdjacentHTML("BeforeEnd",u.join(""))},q.stroke=function(e){var t=[],i=10,n=10;t.push("o.x)&&(o.x=l.x),(null==a.y||l.yo.y)&&(o.y=l.y))}t.push(' ">'),e?k(this,t,a,o):x(this,t),t.push(""),this.element_.insertAdjacentHTML("beforeEnd",t.join(""))},q.fill=function(){this.stroke(!0)},q.closePath=function(){this.currentPath_.push({type:"close"})},q.save=function(){var e={};h(this,e),this.aStack_.push(e),this.mStack_.push(this.m_),this.m_=l(s(),this.m_)},q.restore=function(){this.aStack_.length&&(h(this.aStack_.pop(),this),this.m_=this.mStack_.pop())},q.translate=function(e,t){var i=[[1,0,0],[0,1,0],[e,t,1]];W(this,l(i,this.m_),!1)},q.rotate=function(e){var t=A(e),i=E(e),n=[[t,i,0],[-i,t,0],[0,0,1]];W(this,l(n,this.m_),!1)},q.scale=function(e,t){var i=[[e,0,0],[0,t,0],[0,0,1]];W(this,l(i,this.m_),!0)},q.transform=function(e,t,i,n,a,o){var r=[[e,t,0],[i,n,0],[a,o,1]];W(this,l(r,this.m_),!0)},q.setTransform=function(e,t,i,n,a,o){var r=[[e,t,0],[i,n,0],[a,o,1]];W(this,r,!0)},q.drawText_=function(e,t,n,a,o){var r=this.m_,s=1e3,l=0,h=s,d={x:0,y:0},m=[],c=y(U(this.font),this.element_),p=g(c),u=this.element_.currentStyle,V=this.textAlign.toLowerCase();switch(V){case"left":case"center":case"right":break;case"end":V="ltr"==u.direction?"right":"left";break;case"start":V="rtl"==u.direction?"right":"left";break;default:V="left"}switch(this.textBaseline){case"hanging":case"top":d.y=c.size/1.75;break;case"middle":break;default:case null:case"alphabetic":case"ideographic":case"bottom":d.y=-c.size/2.25}switch(V){case"right":l=s,h=.05;break;case"center":l=h=s/2}var f=L(this,t+d.x,n+d.y);m.push(''),o?x(this,m):k(this,m,{x:-l,y:0},{x:h,y:c.size});var b=r[0][0].toFixed(3)+","+r[1][0].toFixed(3)+","+r[0][1].toFixed(3)+","+r[1][1].toFixed(3)+",0,0",_=C(f.x/J)+","+C(f.y/J);m.push('','',''),this.element_.insertAdjacentHTML("beforeEnd",m.join(""))},q.fillText=function(e,t,i,n){this.drawText_(e,t,i,n,!1)},q.strokeText=function(e,t,i,n){this.drawText_(e,t,i,n,!0)},q.measureText=function(e){if(!this.textMeasureEl_){var t='';this.element_.insertAdjacentHTML("beforeEnd",t),this.textMeasureEl_=this.element_.lastChild}var i=this.element_.ownerDocument;this.textMeasureEl_.innerHTML="";try{this.textMeasureEl_.style.font=this.font}catch(n){}return this.textMeasureEl_.appendChild(i.createTextNode(e)),{width:this.textMeasureEl_.offsetWidth}},q.clip=function(){},q.arcTo=function(){},q.createPattern=function(e,t){return new X(e,t)},w.prototype.addColorStop=function(e,t){t=V(t),this.colors_.push({offset:e,color:t.color,alpha:t.alpha})};var Z=S.prototype=new Error;Z.INDEX_SIZE_ERR=1,Z.DOMSTRING_SIZE_ERR=2,Z.HIERARCHY_REQUEST_ERR=3,Z.WRONG_DOCUMENT_ERR=4,Z.INVALID_CHARACTER_ERR=5,Z.NO_DATA_ALLOWED_ERR=6,Z.NO_MODIFICATION_ALLOWED_ERR=7,Z.NOT_FOUND_ERR=8,Z.NOT_SUPPORTED_ERR=9,Z.INUSE_ATTRIBUTE_ERR=10,Z.INVALID_STATE_ERR=11,Z.SYNTAX_ERR=12,Z.INVALID_MODIFICATION_ERR=13,Z.NAMESPACE_ERR=14,Z.INVALID_ACCESS_ERR=15,Z.VALIDATION_ERR=16,Z.TYPE_MISMATCH_ERR=17,G_vmlCanvasManager=P,CanvasRenderingContext2D=b,CanvasGradient=w,CanvasPattern=X,DOMException=S}(),G_vmlCanvasManager}),define("zrender/mixin/Eventful",["require"],function(){var e=function(){this._handlers={}};return e.prototype.one=function(e,t,i){var n=this._handlers;return t&&e?(n[e]||(n[e]=[]),n[e].push({h:t,one:!0,ctx:i||this}),this):this},e.prototype.bind=function(e,t,i){var n=this._handlers;return t&&e?(n[e]||(n[e]=[]),n[e].push({h:t,one:!1,ctx:i||this}),this):this},e.prototype.unbind=function(e,t){var i=this._handlers;if(!e)return this._handlers={},this;if(t){if(i[e]){for(var n=[],a=0,o=i[e].length;o>a;a++)i[e][a].h!=t&&n.push(i[e][a]);i[e]=n}i[e]&&0===i[e].length&&delete i[e]}else delete i[e];return this},e.prototype.dispatch=function(e){if(this._handlers[e]){var t=arguments,i=t.length;i>3&&(t=Array.prototype.slice.call(t,1));for(var n=this._handlers[e],a=n.length,o=0;a>o;){switch(i){case 1:n[o].h.call(n[o].ctx);break;case 2:n[o].h.call(n[o].ctx,t[1]);break;case 3:n[o].h.call(n[o].ctx,t[1],t[2]); +break;default:n[o].h.apply(n[o].ctx,t)}n[o].one?(n.splice(o,1),a--):o++}}return this},e.prototype.dispatchWithContext=function(e){if(this._handlers[e]){var t=arguments,i=t.length;i>4&&(t=Array.prototype.slice.call(t,1,t.length-1));for(var n=t[t.length-1],a=this._handlers[e],o=a.length,r=0;o>r;){switch(i){case 1:a[r].h.call(n);break;case 2:a[r].h.call(n,t[1]);break;case 3:a[r].h.call(n,t[1],t[2]);break;default:a[r].h.apply(n,t)}a[r].one?(a.splice(r,1),o--):r++}}return this},e}),define("zrender/tool/log",["require","../config"],function(e){var t=e("../config");return function(){if(0!==t.debugMode)if(1==t.debugMode)for(var e in arguments)throw new Error(arguments[e]);else if(t.debugMode>1)for(var e in arguments)console.log(arguments[e])}}),define("zrender/tool/guid",[],function(){var e=2311;return function(){return"zrender__"+e++}}),define("zrender/Handler",["require","./config","./tool/env","./tool/event","./tool/util","./tool/vector","./tool/matrix","./mixin/Eventful"],function(e){"use strict";function t(e,t){return function(i){return e.call(t,i)}}function i(e,t){return function(i,n,a){return e.call(t,i,n,a)}}function n(e){for(var i=p.length;i--;){var n=p[i];e["_"+n+"Handler"]=t(u[n],e)}}function a(e,t,i){if(this._draggingTarget&&this._draggingTarget.id==e.id||e.isSilent())return!1;var n=this._event;if(e.isCover(t,i)){e.hoverable&&this.storage.addHover(e);for(var a=e.parent;a;){if(a.clipShape&&!a.clipShape.isCover(this._mouseX,this._mouseY))return!1;a=a.parent}return this._lastHover!=e&&(this._processOutShape(n),this._processDragLeave(n),this._lastHover=e,this._processDragEnter(n)),this._processOverShape(n),this._processDragOver(n),this._hasfound=1,!0}return!1}var o=e("./config"),r=e("./tool/env"),s=e("./tool/event"),l=e("./tool/util"),h=e("./tool/vector"),d=e("./tool/matrix"),m=o.EVENT,c=e("./mixin/Eventful"),p=["resize","click","dblclick","mousewheel","mousemove","mouseout","mouseup","mousedown","touchstart","touchend","touchmove"],u={resize:function(e){e=e||window.event,this._lastHover=null,this._isMouseDown=0,this.dispatch(m.RESIZE,e)},click:function(e){e=this._zrenderEventFixed(e);var t=this._lastHover;(t&&t.clickable||!t)&&this._clickThreshold<5&&this._dispatchAgency(t,m.CLICK,e),this._mousemoveHandler(e)},dblclick:function(e){e=e||window.event,e=this._zrenderEventFixed(e);var t=this._lastHover;(t&&t.clickable||!t)&&this._clickThreshold<5&&this._dispatchAgency(t,m.DBLCLICK,e),this._mousemoveHandler(e)},mousewheel:function(e){e=this._zrenderEventFixed(e);var t=e.wheelDelta||-e.detail,i=t>0?1.1:1/1.1,n=this.painter.getLayers(),a=!1;for(var o in n)if("hover"!==o){var r=n[o],l=r.position;if(r.zoomable){r.__zoom=r.__zoom||1;var h=r.__zoom;h*=i,h=Math.max(Math.min(r.maxZoom,h),r.minZoom),i=h/r.__zoom,r.__zoom=h,l[0]-=(this._mouseX-l[0])*(i-1),l[1]-=(this._mouseY-l[1])*(i-1),r.scale[0]*=i,r.scale[1]*=i,r.dirty=!0,a=!0,s.stop(e)}}a&&this.painter.refresh(),this._dispatchAgency(this._lastHover,m.MOUSEWHEEL,e),this._mousemoveHandler(e)},mousemove:function(e){if(!this.painter.isLoading()){this._clickThreshold++,e=this._zrenderEventFixed(e),this._lastX=this._mouseX,this._lastY=this._mouseY,this._mouseX=s.getX(e),this._mouseY=s.getY(e);var t=this._mouseX-this._lastX,i=this._mouseY-this._lastY;this._processDragStart(e),this._hasfound=0,this._event=e,this._iterateAndFindHover(),this._hasfound||((!this._draggingTarget||this._lastHover&&this._lastHover!=this._draggingTarget)&&(this._processOutShape(e),this._processDragLeave(e)),this._lastHover=null,this.storage.delHover(),this.painter.clearHover());var n="default";if(this._draggingTarget)this.storage.drift(this._draggingTarget.id,t,i),this._draggingTarget.modSelf(),this.storage.addHover(this._draggingTarget);else if(this._isMouseDown){var a=this.painter.getLayers(),o=!1;for(var r in a)if("hover"!==r){var l=a[r];l.panable&&(n="move",l.position[0]+=t,l.position[1]+=i,o=!0,l.dirty=!0)}o&&this.painter.refresh()}this._draggingTarget||this._hasfound&&this._lastHover.draggable?n="move":this._hasfound&&this._lastHover.clickable&&(n="pointer"),this.root.style.cursor=n,this._dispatchAgency(this._lastHover,m.MOUSEMOVE,e),(this._draggingTarget||this._hasfound||this.storage.hasHoverShape())&&this.painter.refreshHover()}},mouseout:function(e){e=this._zrenderEventFixed(e);var t=e.toElement||e.relatedTarget;if(t!=this.root)for(;t&&9!=t.nodeType;){if(t==this.root)return void this._mousemoveHandler(e);t=t.parentNode}e.zrenderX=this._lastX,e.zrenderY=this._lastY,this.root.style.cursor="default",this._isMouseDown=0,this._processOutShape(e),this._processDrop(e),this._processDragEnd(e),this.painter.isLoading()||this.painter.refreshHover(),this.dispatch(m.GLOBALOUT,e)},mousedown:function(e){return this._clickThreshold=0,2==this._lastDownButton?(this._lastDownButton=e.button,void(this._mouseDownTarget=null)):(this._lastMouseDownMoment=new Date,e=this._zrenderEventFixed(e),this._isMouseDown=1,this._mouseDownTarget=this._lastHover,this._dispatchAgency(this._lastHover,m.MOUSEDOWN,e),void(this._lastDownButton=e.button))},mouseup:function(e){e=this._zrenderEventFixed(e),this.root.style.cursor="default",this._isMouseDown=0,this._clickThreshold=0,this._mouseDownTarget=null,this._dispatchAgency(this._lastHover,m.MOUSEUP,e),this._processDrop(e),this._processDragEnd(e)},touchstart:function(e){e=this._zrenderEventFixed(e,!0),this._lastTouchMoment=new Date,this._mobildFindFixed(e),this._mousedownHandler(e)},touchmove:function(e){e=this._zrenderEventFixed(e,!0),this._mousemoveHandler(e),this._isDragging&&s.stop(e)},touchend:function(e){e=this._zrenderEventFixed(e,!0),this._mouseupHandler(e);var t=new Date;t-this._lastTouchMoment=0;o--){var r=n[o];if(t!==r.zlevel&&(i=this.painter.getLayer(r.zlevel,i),a[0]=this._mouseX,a[1]=this._mouseY,i.needTransform&&(d.invert(e,i.transform),h.applyTransform(a,a,e))),this._findHover(r,a[0],a[1]))break}}}();var U=[{x:10},{x:-20},{x:10,y:10},{y:-20}];return V.prototype._mobildFindFixed=function(e){this._lastHover=null,this._mouseX=e.zrenderX,this._mouseY=e.zrenderY,this._event=e,this._iterateAndFindHover();for(var t=0;!this._lastHover&&td;d++){var p=e[d];if(n!==p.zlevel&&(i&&(i.needTransform&&o.restore(),o.flush&&o.flush()),i=this.getLayer(p.zlevel),o=i.ctx,n=p.zlevel,i.unusedCount=0,(i.dirty||t)&&i.clear(),i.needTransform&&(o.save(),i.setTransform(o))),p.__startClip&&!m){var u=p.__startClip;if(o.save(),u.needTransform){var V=u.transform;s.invert(h,V),o.transform(V[0],V[1],V[2],V[3],V[4],V[5])}if(o.beginPath(),u.buildPath(o,u.style),o.clip(),u.needTransform){var V=h;o.transform(V[0],V[1],V[2],V[3],V[4],V[5])}}if((i.dirty||t)&&!p.invisible&&(!p.onbrush||p.onbrush&&!p.onbrush(o,!1)))if(a.catchBrushException)try{p.brush(o,!1,this.refreshNextFrame)}catch(U){r(U,"brush error of "+p.type,p)}else p.brush(o,!1,this.refreshNextFrame);p.__stopClip&&!m&&o.restore(),p.__dirty=!1}i&&(i.needTransform&&o.restore(),o.flush&&o.flush());for(var l in this._layers)if("hover"!==l){var y=this._layers[l];y.dirty=!1,1==y.unusedCount&&y.clear()}},c.prototype.getLayer=function(e){var t=this._layers[e];if(!t){var i=this._zlevelList.length,n=null,a=-1;if(i>0&&e>this._zlevelList[0]){for(a=0;i-1>a&&!(this._zlevelList[a]e);a++);n=this._layers[this._zlevelList[a]]}this._zlevelList.splice(a+1,0,e),t=new p(e,this);var r=n?n.dom:this._bgDom;r.nextSibling?r.parentNode.insertBefore(t.dom,r.nextSibling):r.parentNode.appendChild(t.dom),t.initContext(),this._layers[e]=t,this._layerConfig[e]&&o.merge(t,this._layerConfig[e],!0),t.updateTransform()}return t},c.prototype.getLayers=function(){return this._layers},c.prototype._updateLayerStatus=function(e){var t=this._layers,i={};for(var n in t)"hover"!==n&&(i[n]=t[n].elCount,t[n].elCount=0);for(var a=0,o=e.length;o>a;a++){var r=e[a],s=r.zlevel,l=t[s];if(l){if(l.elCount++,l.dirty)continue;l.dirty=r.__dirty}}for(var n in t)"hover"!==n&&i[n]!==t[n].elCount&&(t[n].dirty=!0)},c.prototype.refreshShapes=function(e,t){for(var i=0,n=e.length;n>i;i++){var a=e[i];a.modSelf()}return this.refresh(t),this},c.prototype.setLoadingEffect=function(e){return this._loadingEffect=e,this},c.prototype.clear=function(){for(var e in this._layers)"hover"!=e&&this._layers[e].clear();return this},c.prototype.modLayer=function(e,t){if(t){this._layerConfig[e]?o.merge(this._layerConfig[e],t,!0):this._layerConfig[e]=t;var i=this._layers[e];i&&o.merge(i,this._layerConfig[e],!0)}},c.prototype.delLayer=function(e){var t=this._layers[e];t&&(this.modLayer(e,{position:t.position,rotation:t.rotation,scale:t.scale}),t.dom.parentNode.removeChild(t.dom),delete this._layers[e],this._zlevelList.splice(o.indexOf(this._zlevelList,e),1))},c.prototype.refreshHover=function(){this.clearHover();for(var e=this.storage.getHoverShapes(!0),t=0,i=e.length;i>t;t++)this._brushHover(e[t]);var n=this._layers.hover.ctx;return n.flush&&n.flush(),this.storage.delHover(),this},c.prototype.clearHover=function(){var e=this._layers.hover;return e&&e.clear(),this},c.prototype.showLoading=function(e){return this._loadingEffect&&this._loadingEffect.stop(),e&&this.setLoadingEffect(e),this._loadingEffect.start(this),this.loading=!0,this},c.prototype.hideLoading=function(){return this._loadingEffect.stop(),this.clearHover(),this.loading=!1,this},c.prototype.isLoading=function(){return this.loading},c.prototype.resize=function(){var e=this._domRoot;e.style.display="none";var t=this._getWidth(),i=this._getHeight();if(e.style.display="",this._width!=t||i!=this._height){this._width=t,this._height=i,e.style.width=t+"px",e.style.height=i+"px";for(var n in this._layers)this._layers[n].resize(t,i);this.refresh(null,!0)}return this},c.prototype.clearLayer=function(e){var t=this._layers[e];t&&t.clear()},c.prototype.dispose=function(){this.isLoading()&&this.hideLoading(),this.root.innerHTML="",this.root=this.storage=this._domRoot=this._layers=null},c.prototype.getDomHover=function(){return this._layers.hover.dom},c.prototype.toDataURL=function(e,t,i){if(m)return null;var o=n("image","canvas",this);this._bgDom.appendChild(o);var s=o.getContext("2d");1!=d&&s.scale(d,d),s.fillStyle=t||"#fff",s.rect(0,0,this._width*d,this._height*d),s.fill();var l=this;this.storage.iterShape(function(e){if(!e.invisible&&(!e.onbrush||e.onbrush&&!e.onbrush(s,!1)))if(a.catchBrushException)try{e.brush(s,!1,l.refreshNextFrame)}catch(t){r(t,"brush error of "+e.type,e)}else e.brush(s,!1,l.refreshNextFrame)},{normal:"up",update:!0});var h=o.toDataURL(e,i);return s=null,this._bgDom.removeChild(o),h},c.prototype.getWidth=function(){return this._width},c.prototype.getHeight=function(){return this._height},c.prototype._getWidth=function(){var e=this.root,t=e.currentStyle||document.defaultView.getComputedStyle(e);return((e.clientWidth||parseInt(t.width,10))-parseInt(t.paddingLeft,10)-parseInt(t.paddingRight,10)).toFixed(0)-0},c.prototype._getHeight=function(){var e=this.root,t=e.currentStyle||document.defaultView.getComputedStyle(e);return((e.clientHeight||parseInt(t.height,10))-parseInt(t.paddingTop,10)-parseInt(t.paddingBottom,10)).toFixed(0)-0},c.prototype._brushHover=function(e){var t=this._layers.hover.ctx;if(!e.onbrush||e.onbrush&&!e.onbrush(t,!0)){var i=this.getLayer(e.zlevel);if(i.needTransform&&(t.save(),i.setTransform(t)),a.catchBrushException)try{e.brush(t,!0,this.refreshNextFrame)}catch(n){r(n,"hoverBrush error of "+e.type,e)}else e.brush(t,!0,this.refreshNextFrame);i.needTransform&&t.restore()}},c.prototype._shapeToImage=function(t,i,n,a,o){var r=document.createElement("canvas"),s=r.getContext("2d"),o=window.devicePixelRatio||1;r.style.width=n+"px",r.style.height=a+"px",r.setAttribute("width",n*o),r.setAttribute("height",a*o),s.clearRect(0,0,n*o,a*o);var l={position:i.position,rotation:i.rotation,scale:i.scale};i.position=[0,0,0],i.rotation=0,i.scale=[1,1],i&&i.brush(s,!1);var h=e("./shape/Image"),d=new h({id:t,style:{x:0,y:0,image:r}});return null!=l.position&&(d.position=i.position=l.position),null!=l.rotation&&(d.rotation=i.rotation=l.rotation),null!=l.scale&&(d.scale=i.scale=l.scale),d},c.prototype._createShapeToImageProcessor=function(){if(m)return i;var e=this;return function(t,i,n,a){return e._shapeToImage(t,i,n,a,d)}};var p=function(e,i){this.dom=n(e,"canvas",i),this.dom.onselectstart=t,this.dom.style["-webkit-user-select"]="none",this.dom.style["user-select"]="none",this.dom.style["-webkit-touch-callout"]="none",m&&m.initElement(this.dom),this.domBack=null,this.ctxBack=null,this.painter=i,this.unusedCount=0,this.config=null,this.dirty=!0,this.elCount=0,this.clearColor=0,this.motionBlur=!1,this.lastFrameAlpha=.7,this.zoomable=!1,this.panable=!1,this.maxZoom=1/0,this.minZoom=0,h.call(this)};return p.prototype.initContext=function(){this.ctx=this.dom.getContext("2d"),1!=d&&this.ctx.scale(d,d)},p.prototype.createBackBuffer=function(){m||(this.domBack=n("back-"+this.id,"canvas",this.painter),this.ctxBack=this.domBack.getContext("2d"),1!=d&&this.ctxBack.scale(d,d))},p.prototype.resize=function(e,t){this.dom.style.width=e+"px",this.dom.style.height=t+"px",this.dom.setAttribute("width",e*d),this.dom.setAttribute("height",t*d),1!=d&&this.ctx.scale(d,d),this.domBack&&(this.domBack.setAttribute("width",e*d),this.domBack.setAttribute("height",t*d),1!=d&&this.ctxBack.scale(d,d))},p.prototype.clear=function(){var e=this.dom,t=this.ctx,i=e.width,n=e.height,a=this.clearColor&&!m,o=this.motionBlur&&!m,r=this.lastFrameAlpha;if(o&&(this.domBack||this.createBackBuffer(),this.ctxBack.globalCompositeOperation="copy",this.ctxBack.drawImage(e,0,0,i/d,n/d)),a?(t.save(),t.fillStyle=this.config.clearColor,t.fillRect(0,0,i/d,n/d),t.restore()):t.clearRect(0,0,i/d,n/d),o){var s=this.domBack;t.save(),t.globalAlpha=r,t.drawImage(s,0,0,i/d,n/d),t.restore()}},o.merge(p.prototype,h.prototype),c}),define("zrender/Storage",["require","./tool/util","./Group"],function(e){"use strict";function t(e,t){return e.zlevel==t.zlevel?e.z==t.z?e.__renderidx-t.__renderidx:e.z-t.z:e.zlevel-t.zlevel}var i=e("./tool/util"),n=e("./Group"),a={hover:!1,normal:"down",update:!1},o=function(){this._elements={},this._hoverElements=[],this._roots=[],this._shapeList=[],this._shapeListOffset=0};return o.prototype.iterShape=function(e,t){if(t||(t=a),t.hover)for(var i=0,n=this._hoverElements.length;n>i;i++){var o=this._hoverElements[i];if(o.updateTransform(),e(o))return this}switch(t.update&&this.updateShapeList(),t.normal){case"down":for(var n=this._shapeList.length;n--;)if(e(this._shapeList[n]))return this;break;default:for(var i=0,n=this._shapeList.length;n>i;i++)if(e(this._shapeList[i]))return this}return this},o.prototype.getHoverShapes=function(e){for(var i=[],n=0,a=this._hoverElements.length;a>n;n++){i.push(this._hoverElements[n]);var o=this._hoverElements[n].hoverConnect;if(o){var r;o=o instanceof Array?o:[o];for(var s=0,l=o.length;l>s;s++)r=o[s].id?o[s]:this.get(o[s]),r&&i.push(r)}}if(i.sort(t),e)for(var n=0,a=i.length;a>n;n++)i[n].updateTransform();return i},o.prototype.getShapeList=function(e){return e&&this.updateShapeList(),this._shapeList},o.prototype.updateShapeList=function(){this._shapeListOffset=0;for(var e=0,i=this._roots.length;i>e;e++){var n=this._roots[e];this._updateAndAddShape(n)}this._shapeList.length=this._shapeListOffset;for(var e=0,i=this._shapeList.length;i>e;e++)this._shapeList[e].__renderidx=e;this._shapeList.sort(t)},o.prototype._updateAndAddShape=function(e,t){if(!e.ignore)if(e.updateTransform(),"group"==e.type){e.clipShape&&(e.clipShape.parent=e,e.clipShape.updateTransform(),t?(t=t.slice(),t.push(e.clipShape)):t=[e.clipShape]);for(var i=0;i0},o.prototype.addRoot=function(e){e instanceof n&&e.addChildrenToStorage(this),this.addToMap(e),this._roots.push(e)},o.prototype.delRoot=function(e){if("undefined"==typeof e){for(var t=0;tt;t++)this.delRoot(e[t]);else{var r;r="string"==typeof e?this._elements[e]:e;var s=i.indexOf(this._roots,r);s>=0&&(this.delFromMap(r.id),this._roots.splice(s,1),r instanceof n&&r.delChildrenFromStorage(this))}},o.prototype.addToMap=function(e){return e instanceof n&&(e._storage=this),e.modSelf(),this._elements[e.id]=e,this},o.prototype.get=function(e){return this._elements[e]},o.prototype.delFromMap=function(e){var t=this._elements[e];return t&&(delete this._elements[e],t instanceof n&&(t._storage=null)),this},o.prototype.dispose=function(){this._elements=this._renderList=this._roots=this._hoverElements=null},o}),define("zrender/animation/Animation",["require","./Clip","../tool/color","../tool/util","../tool/event"],function(e){"use strict";function t(e,t){return e[t]}function i(e,t,i){e[t]=i}function n(e,t,i){return(t-e)*i+e}function a(e,t,i,a,o){var r=e.length;if(1==o)for(var s=0;r>s;s++)a[s]=n(e[s],t[s],i);else for(var l=e[0].length,s=0;r>s;s++)for(var h=0;l>h;h++)a[s][h]=n(e[s][h],t[s][h],i)}function o(e){switch(typeof e){case"undefined":case"string":return!1}return"undefined"!=typeof e.length}function r(e,t,i,n,a,o,r,l,h){var d=e.length;if(1==h)for(var m=0;d>m;m++)l[m]=s(e[m],t[m],i[m],n[m],a,o,r);else for(var c=e[0].length,m=0;d>m;m++)for(var p=0;c>p;p++)l[m][p]=s(e[m][p],t[m][p],i[m][p],n[m][p],a,o,r)}function s(e,t,i,n,a,o,r){var s=.5*(i-e),l=.5*(n-t);return(2*(t-i)+s+l)*r+(-3*(t-i)-2*s-l)*o+s*a+t}function l(e){if(o(e)){var t=e.length;if(o(e[0])){for(var i=[],n=0;t>n;n++)i.push(V.call(e[n]));return i}return V.call(e)}return e}function h(e){return e[0]=Math.floor(e[0]),e[1]=Math.floor(e[1]),e[2]=Math.floor(e[2]),"rgba("+e.join(",")+")"}var d=e("./Clip"),m=e("../tool/color"),c=e("../tool/util"),p=e("../tool/event").Dispatcher,u=window.requestAnimationFrame||window.msRequestAnimationFrame||window.mozRequestAnimationFrame||window.webkitRequestAnimationFrame||function(e){setTimeout(e,16)},V=Array.prototype.slice,U=function(e){e=e||{},this.stage=e.stage||{},this.onframe=e.onframe||function(){},this._clips=[],this._running=!1,this._time=0,p.call(this)};U.prototype={add:function(e){this._clips.push(e)},remove:function(e){var t=c.indexOf(this._clips,e);t>=0&&this._clips.splice(t,1)},_update:function(){for(var e=(new Date).getTime(),t=e-this._time,i=this._clips,n=i.length,a=[],o=[],r=0;n>r;r++){var s=i[r],l=s.step(e);l&&(a.push(l),o.push(s))}this.stage.update&&this.stage.update();for(var r=0;n>r;)i[r]._needsRemove?(i[r]=i[n-1],i.pop(),n--):r++;n=a.length;for(var r=0;n>r;r++)o[r].fire(a[r]);this._time=e,this.onframe(t),this.dispatch("frame",t)},start:function(){function e(){t._running&&(t._update(),u(e))}var t=this;this._running=!0,this._time=(new Date).getTime(),u(e)},stop:function(){this._running=!1},clear:function(){this._clips=[]},animate:function(e,t){t=t||{};var i=new y(e,t.loop,t.getter,t.setter);return i.animation=this,i},constructor:U},c.merge(U.prototype,p.prototype,!0);var y=function(e,n,a,o){this._tracks={},this._target=e,this._loop=n||!1,this._getter=a||t,this._setter=o||i,this._clipCount=0,this._delay=0,this._doneList=[],this._onframeList=[],this._clipList=[]};return y.prototype={when:function(e,t){for(var i in t)this._tracks[i]||(this._tracks[i]=[],0!==e&&this._tracks[i].push({time:0,value:l(this._getter(this._target,i))})),this._tracks[i].push({time:parseInt(e,10),value:t[i]});return this},during:function(e){return this._onframeList.push(e),this},start:function(e){var t=this,i=this._setter,l=this._getter,c="spline"===e,p=function(){if(t._clipCount--,0===t._clipCount){t._tracks={};for(var e=t._doneList.length,i=0;e>i;i++)t._doneList[i].call(t)}},u=function(u,V){var U=u.length;if(U){var y=u[0].value,g=o(y),f=!1,b=g&&o(y[0])?2:1;u.sort(function(e,t){return e.time-t.time});var _;if(U){_=u[U-1].time;for(var x=[],k=[],L=0;U>L;L++){x.push(u[L].time/_);var v=u[L].value;"string"==typeof v&&(v=m.toArray(v),0===v.length&&(v[0]=v[1]=v[2]=0,v[3]=1),f=!0),k.push(v)}var W,L,w,X,I,K,S,T=0,C=0;if(f)var E=[0,0,0,0];var A=function(e,o){if(C>o){for(W=Math.min(T+1,U-1),L=W;L>=0&&!(x[L]<=o);L--);L=Math.min(L,U-2)}else{for(L=T;U>L&&!(x[L]>o);L++);L=Math.min(L-1,U-2)}T=L,C=o;var d=x[L+1]-x[L];if(0!==d){if(w=(o-x[L])/d,c)if(I=k[L],X=k[0===L?L:L-1],K=k[L>U-2?U-1:L+1],S=k[L>U-3?U-1:L+2],g)r(X,I,K,S,w,w*w,w*w*w,l(e,V),b);else{var m;f?(m=r(X,I,K,S,w,w*w,w*w*w,E,1),m=h(E)):m=s(X,I,K,S,w,w*w,w*w*w),i(e,V,m)}else if(g)a(k[L],k[L+1],w,l(e,V),b);else{var m;f?(a(k[L],k[L+1],w,E,1),m=h(E)):m=n(k[L],k[L+1],w),i(e,V,m)}for(L=0;L=t[1]&&(e=t[1]),e},t.prototype.getLocation=function(e,t,i){var n=null!=e.x?e.x:"center";switch(n){case"center":n=Math.floor((this.canvasWidth-t)/2);break;case"left":n=0;break;case"right":n=this.canvasWidth-t}var a=null!=e.y?e.y:"center";switch(a){case"center":a=Math.floor((this.canvasHeight-i)/2);break;case"top":a=0;break;case"bottom":a=this.canvasHeight-i}return{x:n,y:a,width:t,height:i}},t}),define("zrender/mixin/Transformable",["require","../tool/matrix","../tool/vector"],function(e){"use strict";function t(e){return e>-r&&r>e}function i(e){return e>r||-r>e}var n=e("../tool/matrix"),a=e("../tool/vector"),o=[0,0],r=5e-5,s=function(){this.position||(this.position=[0,0]),"undefined"==typeof this.rotation&&(this.rotation=[0,0,0]),this.scale||(this.scale=[1,1,0,0]),this.needLocalTransform=!1,this.needTransform=!1};return s.prototype={constructor:s,updateNeedTransform:function(){this.needLocalTransform=i(this.rotation[0])||i(this.position[0])||i(this.position[1])||i(this.scale[0]-1)||i(this.scale[1]-1)},updateTransform:function(){if(this.updateNeedTransform(),this.needTransform=this.parent?this.needLocalTransform||this.parent.needTransform:this.needLocalTransform,this.needTransform){var e=this.transform||n.create();if(n.identity(e),this.needLocalTransform){if(i(this.scale[0])||i(this.scale[1])){o[0]=-this.scale[2]||0,o[1]=-this.scale[3]||0;var t=i(o[0])||i(o[1]);t&&n.translate(e,e,o),n.scale(e,e,this.scale),t&&(o[0]=-o[0],o[1]=-o[1],n.translate(e,e,o))}if(this.rotation instanceof Array){if(0!==this.rotation[0]){o[0]=-this.rotation[1]||0,o[1]=-this.rotation[2]||0;var t=i(o[0])||i(o[1]);t&&n.translate(e,e,o),n.rotate(e,e,this.rotation[0]),t&&(o[0]=-o[0],o[1]=-o[1],n.translate(e,e,o))}}else 0!==this.rotation&&n.rotate(e,e,this.rotation);(i(this.position[0])||i(this.position[1]))&&n.translate(e,e,this.position)}this.transform=e,this.parent&&this.parent.needTransform&&(this.needLocalTransform?n.mul(this.transform,this.parent.transform,this.transform):n.copy(this.transform,this.parent.transform))}},setTransform:function(e){if(this.needTransform){var t=this.transform;e.transform(t[0],t[1],t[2],t[3],t[4],t[5])}},lookAt:function(){var e=a.create();return function(i){this.transform||(this.transform=n.create());var o=this.transform;a.sub(e,i,this.position),t(e[0])&&t(e[1])||(a.normalize(e,e),o[2]=e[0]*this.scale[1],o[3]=e[1]*this.scale[1],o[0]=e[1]*this.scale[0],o[1]=-e[0]*this.scale[0],o[4]=this.position[0],o[5]=this.position[1],this.decomposeTransform())}}(),decomposeTransform:function(){if(this.transform){var e=this.transform,t=e[0]*e[0]+e[1]*e[1],n=this.position,a=this.scale,o=this.rotation;i(t-1)&&(t=Math.sqrt(t));var r=e[2]*e[2]+e[3]*e[3];i(r-1)&&(r=Math.sqrt(r)),n[0]=e[4],n[1]=e[5],a[0]=t,a[1]=r,a[2]=a[3]=0,o[0]=Math.atan2(-e[1]/r,e[0]/t),o[1]=o[2]=0}}},s}),define("zrender/shape/Text",["require","../tool/area","./Base","../tool/util"],function(e){var t=e("../tool/area"),i=e("./Base"),n=function(e){i.call(this,e)};return n.prototype={type:"text",brush:function(e,i){var n=this.style;if(i&&(n=this.getHighlightStyle(n,this.highlightStyle||{})),"undefined"!=typeof n.text&&n.text!==!1){e.save(),this.doClip(e),this.setContext(e,n),this.setTransform(e),n.textFont&&(e.font=n.textFont),e.textAlign=n.textAlign||"start",e.textBaseline=n.textBaseline||"middle";var a,o=(n.text+"").split("\n"),r=t.getTextHeight("国",n.textFont),s=this.getRect(n),l=n.x;a="top"==n.textBaseline?s.y:"bottom"==n.textBaseline?s.y+r:s.y+r/2;for(var h=0,d=o.length;d>h;h++){if(n.maxWidth)switch(n.brushType){case"fill":e.fillText(o[h],l,a,n.maxWidth);break;case"stroke":e.strokeText(o[h],l,a,n.maxWidth);break;case"both":e.fillText(o[h],l,a,n.maxWidth),e.strokeText(o[h],l,a,n.maxWidth);break;default:e.fillText(o[h],l,a,n.maxWidth)}else switch(n.brushType){case"fill":e.fillText(o[h],l,a);break;case"stroke":e.strokeText(o[h],l,a);break;case"both":e.fillText(o[h],l,a),e.strokeText(o[h],l,a);break;default:e.fillText(o[h],l,a)}a+=r}e.restore()}},getRect:function(e){if(e.__rect)return e.__rect;var i=t.getTextWidth(e.text,e.textFont),n=t.getTextHeight(e.text,e.textFont),a=e.x;"end"==e.textAlign||"right"==e.textAlign?a-=i:"center"==e.textAlign&&(a-=i/2);var o;return o="top"==e.textBaseline?e.y:"bottom"==e.textBaseline?e.y-n:e.y-n/2,e.__rect={x:a,y:o,width:i,height:n},e.__rect}},e("../tool/util").inherits(n,i),n}),define("zrender/shape/Rectangle",["require","./Base","../tool/util"],function(e){var t=e("./Base"),i=function(e){t.call(this,e)};return i.prototype={type:"rectangle",_buildRadiusPath:function(e,t){var i,n,a,o,r=t.x,s=t.y,l=t.width,h=t.height,d=t.radius;"number"==typeof d?i=n=a=o=d:d instanceof Array?1===d.length?i=n=a=o=d[0]:2===d.length?(i=a=d[0],n=o=d[1]):3===d.length?(i=d[0],n=o=d[1],a=d[2]):(i=d[0],n=d[1],a=d[2],o=d[3]):i=n=a=o=0;var m;i+n>l&&(m=i+n,i*=l/m,n*=l/m),a+o>l&&(m=a+o,a*=l/m,o*=l/m),n+a>h&&(m=n+a,n*=h/m,a*=h/m),i+o>h&&(m=i+o,i*=h/m,o*=h/m),e.moveTo(r+i,s),e.lineTo(r+l-n,s),0!==n&&e.quadraticCurveTo(r+l,s,r+l,s+n),e.lineTo(r+l,s+h-a),0!==a&&e.quadraticCurveTo(r+l,s+h,r+l-a,s+h),e.lineTo(r+o,s+h),0!==o&&e.quadraticCurveTo(r,s+h,r,s+h-o),e.lineTo(r,s+i),0!==i&&e.quadraticCurveTo(r,s,r+i,s)},buildPath:function(e,t){t.radius?this._buildRadiusPath(e,t):(e.moveTo(t.x,t.y),e.lineTo(t.x+t.width,t.y),e.lineTo(t.x+t.width,t.y+t.height),e.lineTo(t.x,t.y+t.height),e.lineTo(t.x,t.y)),e.closePath()},getRect:function(e){if(e.__rect)return e.__rect;var t;return t="stroke"==e.brushType||"fill"==e.brushType?e.lineWidth||1:0,e.__rect={x:Math.round(e.x-t/2),y:Math.round(e.y-t/2),width:e.width+t,height:e.height+t},e.__rect}},e("../tool/util").inherits(i,t),i}),define("zrender/tool/area",["require","./util","./curve"],function(e){"use strict";function t(e){return e%=T,0>e&&(e+=T),e}function i(e,t,i,o){if(!t||!e)return!1;var r=e.type;L=L||v.getContext();var s=n(e,t,i,o);if("undefined"!=typeof s)return s;if(e.buildPath&&L.isPointInPath)return a(e,L,t,i,o);switch(r){case"ellipse":return!0;case"trochoid":var l="out"==t.location?t.r1+t.r2+t.d:t.r1-t.r2+t.d;return p(t,i,o,l);case"rose":return p(t,i,o,t.maxr);default:return!1}}function n(e,t,i,n){var a=e.type;switch(a){case"bezier-curve":return"undefined"==typeof t.cpX2?l(t.xStart,t.yStart,t.cpX1,t.cpY1,t.xEnd,t.yEnd,t.lineWidth,i,n):s(t.xStart,t.yStart,t.cpX1,t.cpY1,t.cpX2,t.cpY2,t.xEnd,t.yEnd,t.lineWidth,i,n);case"line":return r(t.xStart,t.yStart,t.xEnd,t.yEnd,t.lineWidth,i,n);case"broken-line":return d(t.pointList,t.lineWidth,i,n);case"ring":return m(t.x,t.y,t.r0,t.r,i,n);case"circle":return p(t.x,t.y,t.r,i,n);case"sector":var o=t.startAngle*Math.PI/180,h=t.endAngle*Math.PI/180;return t.clockWise||(o=-o,h=-h),u(t.x,t.y,t.r0,t.r,o,h,!t.clockWise,i,n);case"path":return _(t.pathArray,Math.max(t.lineWidth,5),t.brushType,i,n);case"polygon":case"star":case"isogon":return V(t.pointList,i,n);case"text":var U=t.__rect||e.getRect(t);return c(U.x,U.y,U.width,U.height,i,n);case"rectangle":case"image":return c(t.x,t.y,t.width,t.height,i,n)}}function a(e,t,i,n,a){return t.beginPath(),e.buildPath(t,i),t.closePath(),t.isPointInPath(n,a)}function o(e,t,n,a){return!i(e,t,n,a)}function r(e,t,i,n,a,o,r){if(0===a)return!1;var s=Math.max(a,5),l=0,h=e;if(r>t+s&&r>n+s||t-s>r&&n-s>r||o>e+s&&o>i+s||e-s>o&&i-s>o)return!1;if(e===i)return Math.abs(o-e)<=s/2;l=(t-n)/(e-i),h=(e*n-i*t)/(e-i);var d=l*o-r+h,m=d*d/(l*l+1);return s/2*s/2>=m}function s(e,t,i,n,a,o,r,s,l,h,d){if(0===l)return!1;var m=Math.max(l,5);if(d>t+m&&d>n+m&&d>o+m&&d>s+m||t-m>d&&n-m>d&&o-m>d&&s-m>d||h>e+m&&h>i+m&&h>a+m&&h>r+m||e-m>h&&i-m>h&&a-m>h&&r-m>h)return!1;var c=W.cubicProjectPoint(e,t,i,n,a,o,r,s,h,d,null);return m/2>=c}function l(e,t,i,n,a,o,r,s,l){if(0===r)return!1;var h=Math.max(r,5);if(l>t+h&&l>n+h&&l>o+h||t-h>l&&n-h>l&&o-h>l||s>e+h&&s>i+h&&s>a+h||e-h>s&&i-h>s&&a-h>s)return!1;var d=W.quadraticProjectPoint(e,t,i,n,a,o,s,l,null);return h/2>=d}function h(e,i,n,a,o,r,s,l,h){if(0===s)return!1;var d=Math.max(s,5);l-=e,h-=i;var m=Math.sqrt(l*l+h*h);if(m-d>n||n>m+d)return!1;if(Math.abs(a-o)>=T)return!0;if(r){var c=a;a=t(o),o=t(c)}else a=t(a),o=t(o);a>o&&(o+=T);var p=Math.atan2(h,l);return 0>p&&(p+=T),p>=a&&o>=p||p+T>=a&&o>=p+T}function d(e,t,i,n){for(var t=Math.max(t,10),a=0,o=e.length-1;o>a;a++){var s=e[a][0],l=e[a][1],h=e[a+1][0],d=e[a+1][1];if(r(s,l,h,d,t,i,n))return!0}return!1}function m(e,t,i,n,a,o){var r=(a-e)*(a-e)+(o-t)*(o-t);return n*n>r&&r>i*i}function c(e,t,i,n,a,o){return a>=e&&e+i>=a&&o>=t&&t+n>=o}function p(e,t,i,n,a){return i*i>(n-e)*(n-e)+(a-t)*(a-t)}function u(e,t,i,n,a,o,r,s,l){return h(e,t,(i+n)/2,a,o,r,n-i,s,l)}function V(e,t,i){for(var n=e.length,a=0,o=0,r=n-1;n>o;o++){var s=e[r][0],l=e[r][1],h=e[o][0],d=e[o][1];a+=U(s,l,h,d,t,i),r=o}return 0!==a}function U(e,t,i,n,a,o){if(o>t&&o>n||t>o&&n>o)return 0;if(n==t)return 0;var r=t>n?1:-1,s=(o-t)/(n-t),l=s*(i-e)+e;return l>a?r:0}function y(){var e=E[0];E[0]=E[1],E[1]=e}function g(e,t,i,n,a,o,r,s,l,h){if(h>t&&h>n&&h>o&&h>s||t>h&&n>h&&o>h&&s>h)return 0;var d=W.cubicRootAt(t,n,o,s,h,C);if(0===d)return 0;for(var m,c,p=0,u=-1,V=0;d>V;V++){var U=C[V],g=W.cubicAt(e,i,a,r,U);l>g||(0>u&&(u=W.cubicExtrema(t,n,o,s,E),E[1]1&&y(),m=W.cubicAt(t,n,o,s,E[0]),u>1&&(c=W.cubicAt(t,n,o,s,E[1]))),p+=2==u?Um?1:-1:Uc?1:-1:c>s?1:-1:Um?1:-1:m>s?1:-1)}return p}function f(e,t,i,n,a,o,r,s){if(s>t&&s>n&&s>o||t>s&&n>s&&o>s)return 0;var l=W.quadraticRootAt(t,n,o,s,C);if(0===l)return 0;var h=W.quadraticExtremum(t,n,o);if(h>=0&&1>=h){for(var d=0,m=W.quadraticAt(t,n,o,h),c=0;l>c;c++){var p=W.quadraticAt(e,i,a,C[c]);p>r||(d+=C[c]m?1:-1:m>o?1:-1)}return d}var p=W.quadraticAt(e,i,a,C[0]);return p>r?0:t>o?1:-1}function b(e,i,n,a,o,r,s,l){if(l-=i,l>n||-n>l)return 0;var h=Math.sqrt(n*n-l*l);if(C[0]=-h,C[1]=h,Math.abs(a-o)>=T){a=0,o=T;var d=r?1:-1;return s>=C[0]+e&&s<=C[1]+e?d:0}if(r){var h=a;a=t(o),o=t(h)}else a=t(a),o=t(o);a>o&&(o+=T);for(var m=0,c=0;2>c;c++){var p=C[c];if(p+e>s){var u=Math.atan2(l,p),d=r?1:-1;0>u&&(u=T+u),(u>=a&&o>=u||u+T>=a&&o>=u+T)&&(u>Math.PI/2&&u<1.5*Math.PI&&(d=-d),m+=d)}}return m}function _(e,t,i,n,a){var o=0,d=0,m=0,c=0,p=0,u=!0,V=!0;i=i||"fill";for(var y="stroke"===i||"both"===i,_="fill"===i||"both"===i,x=0;x0&&(_&&(o+=U(d,m,c,p,n,a)),0!==o))return!0;c=L[L.length-2],p=L[L.length-1],u=!1,V&&"A"!==k.command&&(V=!1,d=c,m=p)}switch(k.command){case"M":d=L[0],m=L[1];break;case"L":if(y&&r(d,m,L[0],L[1],t,n,a))return!0;_&&(o+=U(d,m,L[0],L[1],n,a)),d=L[0],m=L[1];break;case"C":if(y&&s(d,m,L[0],L[1],L[2],L[3],L[4],L[5],t,n,a))return!0;_&&(o+=g(d,m,L[0],L[1],L[2],L[3],L[4],L[5],n,a)),d=L[4],m=L[5];break;case"Q":if(y&&l(d,m,L[0],L[1],L[2],L[3],t,n,a))return!0;_&&(o+=f(d,m,L[0],L[1],L[2],L[3],n,a)),d=L[2],m=L[3];break;case"A":var v=L[0],W=L[1],w=L[2],X=L[3],I=L[4],K=L[5],S=Math.cos(I)*w+v,T=Math.sin(I)*X+W;V?(V=!1,c=S,p=T):o+=U(d,m,S,T);var C=(n-v)*X/w+v;if(y&&h(v,W,X,I,I+K,1-L[7],t,C,a))return!0;_&&(o+=b(v,W,X,I,I+K,1-L[7],C,a)),d=Math.cos(I+K)*w+v,m=Math.sin(I+K)*X+W;break;case"z":if(y&&r(d,m,c,p,t,n,a))return!0;u=!0}}return _&&(o+=U(d,m,c,p,n,a)),0!==o}function x(e,t){var i=e+":"+t;if(w[i])return w[i];L=L||v.getContext(),L.save(),t&&(L.font=t),e=(e+"").split("\n");for(var n=0,a=0,o=e.length;o>a;a++)n=Math.max(L.measureText(e[a]).width,n);return L.restore(),w[i]=n,++I>S&&(I=0,w={}),n}function k(e,t){var i=e+":"+t;if(X[i])return X[i];L=L||v.getContext(),L.save(),t&&(L.font=t),e=(e+"").split("\n");var n=(L.measureText("国").width+2)*e.length;return L.restore(),X[i]=n,++K>S&&(K=0,X={}),n}var L,v=e("./util"),W=e("./curve"),w={},X={},I=0,K=0,S=5e3,T=2*Math.PI,C=[-1,-1,-1],E=[-1,-1];return{isInside:i,isOutside:o,getTextWidth:x,getTextHeight:k,isInsidePath:_,isInsidePolygon:V,isInsideSector:u,isInsideCircle:p,isInsideLine:r,isInsideRect:c,isInsideBrokenLine:d,isInsideCubicStroke:s,isInsideQuadraticStroke:l}}),define("zrender/shape/Base",["require","../tool/matrix","../tool/guid","../tool/util","../tool/log","../mixin/Transformable","../mixin/Eventful","../tool/area","../tool/color"],function(e){function t(t,n,a,o,r,s,l){r&&(t.font=r),t.textAlign=s,t.textBaseline=l;var h=i(n,a,o,r,s,l);n=(n+"").split("\n");var d=e("../tool/area").getTextHeight("国",r);switch(l){case"top":o=h.y;break;case"bottom":o=h.y+d;break;default:o=h.y+d/2}for(var m=0,c=n.length;c>m;m++)t.fillText(n[m],a,o),o+=d}function i(t,i,n,a,o,r){var s=e("../tool/area"),l=s.getTextWidth(t,a),h=s.getTextHeight("国",a);switch(t=(t+"").split("\n"),o){case"end":case"right":i-=l;break;case"center":i-=l/2}switch(r){case"top":break;case"bottom":n-=h*t.length;break;default:n-=h*t.length/2}return{x:i,y:n,width:l,height:h*t.length}}var n=window.G_vmlCanvasManager,a=e("../tool/matrix"),o=e("../tool/guid"),r=e("../tool/util"),s=e("../tool/log"),l=e("../mixin/Transformable"),h=e("../mixin/Eventful"),d=function(e){e=e||{},this.id=e.id||o();for(var t in e)this[t]=e[t];this.style=this.style||{},this.highlightStyle=this.highlightStyle||null,this.parent=null,this.__dirty=!0,this.__clipShapes=[],l.call(this),h.call(this)};d.prototype.invisible=!1,d.prototype.ignore=!1,d.prototype.zlevel=0,d.prototype.draggable=!1,d.prototype.clickable=!1,d.prototype.hoverable=!0,d.prototype.z=0,d.prototype.brush=function(e,t){var i=this.beforeBrush(e,t);switch(e.beginPath(),this.buildPath(e,i),i.brushType){case"both":e.fill();case"stroke":i.lineWidth>0&&e.stroke();break;default:e.fill()}this.drawText(e,i,this.style),this.afterBrush(e)},d.prototype.beforeBrush=function(e,t){var i=this.style;return this.brushTypeOnly&&(i.brushType=this.brushTypeOnly),t&&(i=this.getHighlightStyle(i,this.highlightStyle||{},this.brushTypeOnly)),"stroke"==this.brushTypeOnly&&(i.strokeColor=i.strokeColor||i.color),e.save(),this.doClip(e),this.setContext(e,i),this.setTransform(e),i},d.prototype.afterBrush=function(e){e.restore()};var m=[["color","fillStyle"],["strokeColor","strokeStyle"],["opacity","globalAlpha"],["lineCap","lineCap"],["lineJoin","lineJoin"],["miterLimit","miterLimit"],["lineWidth","lineWidth"],["shadowBlur","shadowBlur"],["shadowColor","shadowColor"],["shadowOffsetX","shadowOffsetX"],["shadowOffsetY","shadowOffsetY"]];d.prototype.setContext=function(e,t){for(var i=0,n=m.length;n>i;i++){var a=m[i][0],o=t[a],r=m[i][1];"undefined"!=typeof o&&(e[r]=o)}};var c=a.create();return d.prototype.doClip=function(e){if(this.__clipShapes&&!n)for(var t=0;t=a.x&&t<=a.x+a.width&&i>=a.y&&i<=a.y+a.height?e("../tool/area").isInside(this,this.style,t,i):!1},d.prototype.drawText=function(e,i,n){if("undefined"!=typeof i.text&&i.text!==!1){var a=i.textColor||i.color||i.strokeColor;e.fillStyle=a;var o,r,s,l,h=10,d=i.textPosition||this.textPosition||"top";switch(d){case"inside":case"top":case"bottom":case"left":case"right":if(this.getRect){var m=(n||i).__rect||this.getRect(n||i);switch(d){case"inside":s=m.x+m.width/2,l=m.y+m.height/2,o="center",r="middle","stroke"!=i.brushType&&a==i.color&&(e.fillStyle="#fff");break;case"left":s=m.x-h,l=m.y+m.height/2,o="end",r="middle";break;case"right":s=m.x+m.width+h,l=m.y+m.height/2,o="start",r="middle";break;case"top":s=m.x+m.width/2,l=m.y-h,o="center",r="bottom";break;case"bottom":s=m.x+m.width/2,l=m.y+m.height+h,o="center",r="top"}}break;case"start":case"end":var c,p,u,V;if("undefined"!=typeof i.pointList){var U=i.pointList;if(U.length<2)return;var y=U.length;switch(d){case"start":c=U[0][0],p=U[1][0],u=U[0][1],V=U[1][1];break;case"end":c=U[y-2][0],p=U[y-1][0],u=U[y-2][1],V=U[y-1][1]}}else c=i.xStart||0,p=i.xEnd||0,u=i.yStart||0,V=i.yEnd||0;switch(d){case"start":o=p>c?"end":"start",r=V>u?"bottom":"top",s=c,l=u;break;case"end":o=p>c?"start":"end",r=V>u?"top":"bottom",s=p,l=V}h-=4,c!=p?s-="end"==o?h:-h:o="center",u!=V?l-="bottom"==r?h:-h:r="middle";break;case"specific":s=i.textX||0,l=i.textY||0,o="start",r="middle"}null!=s&&null!=l&&t(e,i.text,s,l,i.textFont,i.textAlign||o,i.textBaseline||r)}},d.prototype.modSelf=function(){this.__dirty=!0,this.style&&(this.style.__rect=null),this.highlightStyle&&(this.highlightStyle.__rect=null)},d.prototype.isSilent=function(){return!(this.hoverable||this.draggable||this.clickable||this.onmousemove||this.onmouseover||this.onmouseout||this.onmousedown||this.onmouseup||this.onclick||this.ondragenter||this.ondragover||this.ondragleave||this.ondrop)},r.merge(d.prototype,l.prototype,!0),r.merge(d.prototype,h.prototype,!0),d}),define("zrender/tool/curve",["require","./vector"],function(e){function t(e){return e>-V&&V>e}function i(e){return e>V||-V>e}function n(e,t,i,n,a){var o=1-a;return o*o*(o*e+3*a*t)+a*a*(a*n+3*o*i)}function a(e,t,i,n,a){var o=1-a;return 3*(((t-e)*o+2*(i-t)*a)*o+(n-i)*a*a)}function o(e,i,n,a,o,r){var s=a+3*(i-n)-e,l=3*(n-2*i+e),h=3*(i-e),d=e-o,m=l*l-3*s*h,c=l*h-9*s*d,p=h*h-3*l*d,u=0;if(t(m)&&t(c))if(t(l))r[0]=0;else{var V=-h/l;V>=0&&1>=V&&(r[u++]=V)}else{var g=c*c-4*m*p;if(t(g)){var f=c/m,V=-l/s+f,b=-f/2;V>=0&&1>=V&&(r[u++]=V),b>=0&&1>=b&&(r[u++]=b)}else if(g>0){var _=Math.sqrt(g),x=m*l+1.5*s*(-c+_),k=m*l+1.5*s*(-c-_);x=0>x?-Math.pow(-x,y):Math.pow(x,y),k=0>k?-Math.pow(-k,y):Math.pow(k,y);var V=(-l-(x+k))/(3*s);V>=0&&1>=V&&(r[u++]=V)}else{var L=(2*m*l-3*s*c)/(2*Math.sqrt(m*m*m)),v=Math.acos(L)/3,W=Math.sqrt(m),w=Math.cos(v),V=(-l-2*W*w)/(3*s),b=(-l+W*(w+U*Math.sin(v)))/(3*s),X=(-l+W*(w-U*Math.sin(v)))/(3*s);V>=0&&1>=V&&(r[u++]=V),b>=0&&1>=b&&(r[u++]=b),X>=0&&1>=X&&(r[u++]=X)}}return u}function r(e,n,a,o,r){var s=6*a-12*n+6*e,l=9*n+3*o-3*e-9*a,h=3*n-3*e,d=0;if(t(l)){if(i(s)){var m=-h/s;m>=0&&1>=m&&(r[d++]=m)}}else{var c=s*s-4*l*h;if(t(c))r[0]=-s/(2*l);else if(c>0){var p=Math.sqrt(c),m=(-s+p)/(2*l),u=(-s-p)/(2*l);m>=0&&1>=m&&(r[d++]=m),u>=0&&1>=u&&(r[d++]=u)}}return d}function s(e,t,i,n,a,o){var r=(t-e)*a+e,s=(i-t)*a+t,l=(n-i)*a+i,h=(s-r)*a+r,d=(l-s)*a+s,m=(d-h)*a+h;o[0]=e,o[1]=r,o[2]=h,o[3]=m,o[4]=m,o[5]=d,o[6]=l,o[7]=n}function l(e,t,i,a,o,r,s,l,h,d,m){var c,p=.005,U=1/0;g[0]=h,g[1]=d;for(var y=0;1>y;y+=.05){f[0]=n(e,i,o,s,y),f[1]=n(t,a,r,l,y);var _=u.distSquare(g,f);U>_&&(c=y,U=_)}U=1/0;for(var x=0;32>x&&!(V>p);x++){var k=c-p,L=c+p;f[0]=n(e,i,o,s,k),f[1]=n(t,a,r,l,k);var _=u.distSquare(f,g);if(k>=0&&U>_)c=k,U=_;else{b[0]=n(e,i,o,s,L),b[1]=n(t,a,r,l,L);var v=u.distSquare(b,g);1>=L&&U>v?(c=L,U=v):p*=.5}}return m&&(m[0]=n(e,i,o,s,c),m[1]=n(t,a,r,l,c)),Math.sqrt(U)}function h(e,t,i,n){var a=1-n;return a*(a*e+2*n*t)+n*n*i}function d(e,t,i,n){return 2*((1-n)*(t-e)+n*(i-t))}function m(e,n,a,o,r){var s=e-2*n+a,l=2*(n-e),h=e-o,d=0;if(t(s)){if(i(l)){var m=-h/l;m>=0&&1>=m&&(r[d++]=m)}}else{var c=l*l-4*s*h;if(t(c)){var m=-l/(2*s);m>=0&&1>=m&&(r[d++]=m)}else if(c>0){var p=Math.sqrt(c),m=(-l+p)/(2*s),u=(-l-p)/(2*s);m>=0&&1>=m&&(r[d++]=m),u>=0&&1>=u&&(r[d++]=u)}}return d}function c(e,t,i){var n=e+i-2*t;return 0===n?.5:(e-t)/n}function p(e,t,i,n,a,o,r,s,l){var d,m=.005,c=1/0;g[0]=r,g[1]=s;for(var p=0;1>p;p+=.05){f[0]=h(e,i,a,p),f[1]=h(t,n,o,p);var U=u.distSquare(g,f);c>U&&(d=p,c=U)}c=1/0;for(var y=0;32>y&&!(V>m);y++){var _=d-m,x=d+m;f[0]=h(e,i,a,_),f[1]=h(t,n,o,_);var U=u.distSquare(f,g);if(_>=0&&c>U)d=_,c=U;else{b[0]=h(e,i,a,x),b[1]=h(t,n,o,x);var k=u.distSquare(b,g);1>=x&&c>k?(d=x,c=k):m*=.5}}return l&&(l[0]=h(e,i,a,d),l[1]=h(t,n,o,d)),Math.sqrt(c)}var u=e("./vector"),V=1e-4,U=Math.sqrt(3),y=1/3,g=u.create(),f=u.create(),b=u.create();return{cubicAt:n,cubicDerivativeAt:a,cubicRootAt:o,cubicExtrema:r,cubicSubdivide:s,cubicProjectPoint:l,quadraticAt:h,quadraticDerivativeAt:d,quadraticRootAt:m,quadraticExtremum:c,quadraticProjectPoint:p}}),define("zrender/Group",["require","./tool/guid","./tool/util","./mixin/Transformable","./mixin/Eventful"],function(e){var t=e("./tool/guid"),i=e("./tool/util"),n=e("./mixin/Transformable"),a=e("./mixin/Eventful"),o=function(e){e=e||{},this.id=e.id||t();for(var i in e)this[i]=e[i];this.type="group",this.clipShape=null,this._children=[],this._storage=null,this.__dirty=!0,n.call(this),a.call(this)};return o.prototype.ignore=!1,o.prototype.children=function(){return this._children.slice()},o.prototype.childAt=function(e){return this._children[e]},o.prototype.addChild=function(e){e!=this&&e.parent!=this&&(e.parent&&e.parent.removeChild(e),this._children.push(e),e.parent=this,this._storage&&this._storage!==e._storage&&(this._storage.addToMap(e),e instanceof o&&e.addChildrenToStorage(this._storage)))},o.prototype.removeChild=function(e){var t=i.indexOf(this._children,e);this._children.splice(t,1),e.parent=null,this._storage&&(this._storage.delFromMap(e.id),e instanceof o&&e.delChildrenFromStorage(this._storage))},o.prototype.eachChild=function(e,t){for(var i=!!t,n=0;nt)){t=Math.min(t,1);var n="string"==typeof this.easing?i[this.easing]:this.easing,a="function"==typeof n?n(t):t;return this.fire("frame",a),1==t?this.loop?(this.restart(),"restart"):(this._needsRemove=!0,"destroy"):null}},restart:function(){var e=(new Date).getTime(),t=(e-this._startTime)%this._life;this._startTime=(new Date).getTime()-t+this.gap,this._needsRemove=!1},fire:function(e,t){for(var i=0,n=this._targetPool.length;n>i;i++)this["on"+e]&&this["on"+e](this._targetPool[i],t)},constructor:t},t}),define("zrender/animation/easing",[],function(){var e={Linear:function(e){return e},QuadraticIn:function(e){return e*e},QuadraticOut:function(e){return e*(2-e)},QuadraticInOut:function(e){return(e*=2)<1?.5*e*e:-.5*(--e*(e-2)-1)},CubicIn:function(e){return e*e*e},CubicOut:function(e){return--e*e*e+1},CubicInOut:function(e){return(e*=2)<1?.5*e*e*e:.5*((e-=2)*e*e+2)},QuarticIn:function(e){return e*e*e*e},QuarticOut:function(e){return 1- --e*e*e*e},QuarticInOut:function(e){return(e*=2)<1?.5*e*e*e*e:-.5*((e-=2)*e*e*e-2)},QuinticIn:function(e){return e*e*e*e*e},QuinticOut:function(e){return--e*e*e*e*e+1},QuinticInOut:function(e){return(e*=2)<1?.5*e*e*e*e*e:.5*((e-=2)*e*e*e*e+2)},SinusoidalIn:function(e){return 1-Math.cos(e*Math.PI/2)},SinusoidalOut:function(e){return Math.sin(e*Math.PI/2)},SinusoidalInOut:function(e){return.5*(1-Math.cos(Math.PI*e))},ExponentialIn:function(e){return 0===e?0:Math.pow(1024,e-1)},ExponentialOut:function(e){return 1===e?1:1-Math.pow(2,-10*e)},ExponentialInOut:function(e){return 0===e?0:1===e?1:(e*=2)<1?.5*Math.pow(1024,e-1):.5*(-Math.pow(2,-10*(e-1))+2)},CircularIn:function(e){return 1-Math.sqrt(1-e*e)},CircularOut:function(e){return Math.sqrt(1- --e*e)},CircularInOut:function(e){return(e*=2)<1?-.5*(Math.sqrt(1-e*e)-1):.5*(Math.sqrt(1-(e-=2)*e)+1)},ElasticIn:function(e){var t,i=.1,n=.4;return 0===e?0:1===e?1:(!i||1>i?(i=1,t=n/4):t=n*Math.asin(1/i)/(2*Math.PI),-(i*Math.pow(2,10*(e-=1))*Math.sin(2*(e-t)*Math.PI/n)))},ElasticOut:function(e){var t,i=.1,n=.4;return 0===e?0:1===e?1:(!i||1>i?(i=1,t=n/4):t=n*Math.asin(1/i)/(2*Math.PI),i*Math.pow(2,-10*e)*Math.sin(2*(e-t)*Math.PI/n)+1)},ElasticInOut:function(e){var t,i=.1,n=.4;return 0===e?0:1===e?1:(!i||1>i?(i=1,t=n/4):t=n*Math.asin(1/i)/(2*Math.PI),(e*=2)<1?-.5*i*Math.pow(2,10*(e-=1))*Math.sin(2*(e-t)*Math.PI/n):i*Math.pow(2,-10*(e-=1))*Math.sin(2*(e-t)*Math.PI/n)*.5+1)},BackIn:function(e){var t=1.70158;return e*e*((t+1)*e-t)},BackOut:function(e){var t=1.70158;return--e*e*((t+1)*e+t)+1},BackInOut:function(e){var t=2.5949095;return(e*=2)<1?.5*e*e*((t+1)*e-t):.5*((e-=2)*e*((t+1)*e+t)+2)},BounceIn:function(t){return 1-e.BounceOut(1-t)},BounceOut:function(e){return 1/2.75>e?7.5625*e*e:2/2.75>e?7.5625*(e-=1.5/2.75)*e+.75:2.5/2.75>e?7.5625*(e-=2.25/2.75)*e+.9375:7.5625*(e-=2.625/2.75)*e+.984375},BounceInOut:function(t){return.5>t?.5*e.BounceIn(2*t):.5*e.BounceOut(2*t-1)+.5}};return e}),define("echarts/component/base",["require","../config","../util/ecData","../util/ecQuery","../util/number","zrender/tool/util","zrender/tool/env"],function(e){function t(e,t,a,o,r){this.ecTheme=e,this.messageCenter=t,this.zr=a,this.option=o,this.series=o.series,this.myChart=r,this.component=r.component,this._zlevelBase=this.getZlevelBase(),this.shapeList=[],this.effectList=[];var s=this;s._onlegendhoverlink=function(e){if(s.legendHoverLink)for(var t,a=e.target,o=s.shapeList.length-1;o>=0;o--)t=s.type==i.CHART_TYPE_PIE||s.type==i.CHART_TYPE_FUNNEL?n.get(s.shapeList[o],"name"):(n.get(s.shapeList[o],"series")||{}).name,t!=a||s.shapeList[o].invisible||s.zr.addHoverShape(s.shapeList[o])},t&&t.bind(i.EVENT.LEGEND_HOVERLINK,this._onlegendhoverlink)}var i=e("../config"),n=e("../util/ecData"),a=e("../util/ecQuery"),o=e("../util/number"),r=e("zrender/tool/util");return t.prototype={canvasSupported:e("zrender/tool/env").canvasSupported,getZlevelBase:function(e){switch(e=e||this.type+""){case i.COMPONENT_TYPE_GRID:case i.COMPONENT_TYPE_AXIS_CATEGORY:case i.COMPONENT_TYPE_AXIS_VALUE:case i.COMPONENT_TYPE_POLAR:return 0;case i.CHART_TYPE_LINE:case i.CHART_TYPE_BAR:case i.CHART_TYPE_SCATTER:case i.CHART_TYPE_PIE:case i.CHART_TYPE_RADAR:case i.CHART_TYPE_MAP:case i.CHART_TYPE_K:case i.CHART_TYPE_CHORD:case i.CHART_TYPE_GUAGE:case i.CHART_TYPE_FUNNEL:case i.CHART_TYPE_EVENTRIVER:return 2;case i.COMPONENT_TYPE_LEGEND:case i.COMPONENT_TYPE_DATARANGE:case i.COMPONENT_TYPE_DATAZOOM:case i.COMPONENT_TYPE_TIMELINE:case i.COMPONENT_TYPE_ROAMCONTROLLER:return 4;case i.CHART_TYPE_ISLAND:return 5;case i.COMPONENT_TYPE_TOOLBOX:case i.COMPONENT_TYPE_TITLE:return 6;case i.COMPONENT_TYPE_TOOLTIP:return 8;default:return 0}},reformOption:function(e){return r.merge(e||{},r.clone(this.ecTheme[this.type]||{}))},reformCssArray:function(e){if(!(e instanceof Array))return[e,e,e,e];switch(e.length+""){case"4":return e;case"3":return[e[0],e[1],e[2],e[1]];case"2":return[e[0],e[1],e[0],e[1]];case"1":return[e[0],e[0],e[0],e[0]];case"0":return[0,0,0,0]}},getShapeById:function(e){for(var t=0,i=this.shapeList.length;i>t;t++)if(this.shapeList[t].id===e)return this.shapeList[t];return null},getFont:function(e){var t=r.merge(r.clone(e)||{},this.ecTheme.textStyle);return t.fontStyle+" "+t.fontWeight+" "+t.fontSize+"px "+t.fontFamily},getItemStyleColor:function(e,t,i,n){return"function"==typeof e?e.call(this.myChart,{seriesIndex:t,series:this.series[t],dataIndex:i,data:n}):e},subPixelOptimize:function(e,t){return e=t%2===1?Math.floor(e)+.5:Math.round(e)},resize:function(){this.refresh&&this.refresh(),this.clearEffectShape&&this.clearEffectShape(!0);var e=this;setTimeout(function(){e.animationEffect&&e.animationEffect()},200)},clear:function(){this.clearEffectShape&&this.clearEffectShape(),this.zr&&this.zr.delShape(this.shapeList),this.shapeList=[]},dispose:function(){this.onbeforDispose&&this.onbeforDispose(),this.clear(),this.shapeList=null,this.effectList=null,this.messageCenter&&this.messageCenter.unbind(i.EVENT.LEGEND_HOVERLINK,this._onlegendhoverlink),this.onafterDispose&&this.onafterDispose()},query:a.query,deepQuery:a.deepQuery,deepMerge:a.deepMerge,parsePercent:o.parsePercent,parseCenter:o.parseCenter,parseRadius:o.parseRadius,numAddCommas:o.addCommas},t}),define("echarts/chart/base",["require","zrender/shape/Image","../util/shape/Icon","../util/shape/MarkLine","../util/shape/Symbol","../config","../util/ecData","../util/ecAnimation","../util/ecEffect","../util/accMath","zrender/tool/util","zrender/tool/area"],function(e){function t(){var e=this;this.selectedMap={},this.lastShapeList=[],this.shapeHandler={onclick:function(){e.isClick=!0},ondragover:function(t){var i=t.target;i.highlightStyle=i.highlightStyle||{};var n=i.highlightStyle,a=n.brushTyep,o=n.strokeColor,r=n.lineWidth;n.brushType="stroke",n.strokeColor=e.ecTheme.calculableColor,n.lineWidth="icon"===i.type?30:10,e.zr.addHoverShape(i),setTimeout(function(){i.highlightStyle&&(i.highlightStyle.brushType=a,i.highlightStyle.strokeColor=o,i.highlightStyle.lineWidth=r)},20)},ondrop:function(t){null!=s.get(t.dragged,"data")&&(e.isDrop=!0)},ondragend:function(){e.isDragend=!0}}}var i=e("zrender/shape/Image"),n=e("../util/shape/Icon"),a=e("../util/shape/MarkLine"),o=e("../util/shape/Symbol"),r=e("../config"),s=e("../util/ecData"),l=e("../util/ecAnimation"),h=e("../util/ecEffect"),d=e("../util/accMath"),m=e("zrender/tool/util"),c=e("zrender/tool/area");return t.prototype={setCalculable:function(e){return e.dragEnableTime=this.ecTheme.DRAG_ENABLE_TIME,e.ondragover=this.shapeHandler.ondragover,e.ondragend=this.shapeHandler.ondragend,e.ondrop=this.shapeHandler.ondrop,e +},ondrop:function(e,t){if(this.isDrop&&e.target&&!t.dragIn){var i,n=e.target,a=e.dragged,o=s.get(n,"seriesIndex"),l=s.get(n,"dataIndex"),h=this.series,m=this.component.legend;if(-1===l){if(s.get(a,"seriesIndex")==o)return t.dragOut=t.dragIn=t.needRefresh=!0,void(this.isDrop=!1);i={value:s.get(a,"value"),name:s.get(a,"name")},this.type===r.CHART_TYPE_PIE&&i.value<0&&(i.value=0);for(var c=!1,p=h[o].data,u=0,V=p.length;V>u;u++)p[u].name===i.name&&"-"===p[u].value&&(h[o].data[u].value=i.value,c=!0);!c&&h[o].data.push(i),m&&m.add(i.name,a.style.color||a.style.strokeColor)}else i=this.option.series[o].data[l]||"-",null!=i.value?(this.option.series[o].data[l].value="-"!=i.value?d.accAdd(this.option.series[o].data[l].value,s.get(a,"value")):s.get(a,"value"),(this.type===r.CHART_TYPE_FUNNEL||this.type===r.CHART_TYPE_PIE)&&(m&&1===m.getRelatedAmount(i.name)&&this.component.legend.del(i.name),i.name+=this.option.nameConnector+s.get(a,"name"),m&&m.add(i.name,a.style.color||a.style.strokeColor))):this.option.series[o].data[l]="-"!=i?d.accAdd(this.option.series[o].data[l],s.get(a,"value")):s.get(a,"value");t.dragIn=t.dragIn||!0,this.isDrop=!1;var U=this;setTimeout(function(){U.zr.trigger("mousemove",e.event)},300)}},ondragend:function(e,t){if(this.isDragend&&e.target&&!t.dragOut){var i=e.target,n=s.get(i,"seriesIndex"),a=s.get(i,"dataIndex"),o=this.series;if(null!=o[n].data[a].value){o[n].data[a].value="-";var r=o[n].data[a].name;this.component.legend&&0===this.component.legend.getRelatedAmount(r)&&this.component.legend.del(r)}else o[n].data[a]="-";t.dragOut=!0,t.needRefresh=!0,this.isDragend=!1}},onlegendSelected:function(e,t){var i=e.selected;for(var n in this.selectedMap)this.selectedMap[n]!=i[n]&&(t.needRefresh=!0),this.selectedMap[n]=i[n]},_bulidPosition:function(){this._symbol=this.option.symbolList,this._sIndex2ShapeMap={},this._sIndex2ColorMap={},this.selectedMap={},this.xMarkMap={};for(var e,t,i,n,a=this.series,o={top:[],bottom:[],left:[],right:[],other:[]},s=0,l=a.length;l>s;s++)a[s].type===this.type&&(a[s]=this.reformOption(a[s]),this.legendHoverLink=a[s].legendHoverLink||this.legendHoverLink,e=a[s].xAxisIndex,t=a[s].yAxisIndex,i=this.component.xAxis.getAxis(e),n=this.component.yAxis.getAxis(t),i.type===r.COMPONENT_TYPE_AXIS_CATEGORY?o[i.getPosition()].push(s):n.type===r.COMPONENT_TYPE_AXIS_CATEGORY?o[n.getPosition()].push(s):o.other.push(s));for(var h in o)o[h].length>0&&this._buildSinglePosition(h,o[h]);this.addShapeList()},_buildSinglePosition:function(e,t){var i=this._mapData(t),n=i.locationMap,a=i.maxDataLength;if(0!==a&&0!==n.length){switch(e){case"bottom":case"top":this._buildHorizontal(t,a,n,this.xMarkMap);break;case"left":case"right":this._buildVertical(t,a,n,this.xMarkMap);break;case"other":this._buildOther(t,a,n,this.xMarkMap)}for(var o=0,r=t.length;r>o;o++)this.buildMark(t[o])}},_mapData:function(e){for(var t,i,n,a,o=this.series,s=0,l={},h="__kener__stack__",d=this.component.legend,m=[],c=0,p=0,u=e.length;u>p;p++)t=o[e[p]],n=t.name,this._sIndex2ShapeMap[e[p]]=this._sIndex2ShapeMap[e[p]]||this.query(t,"symbol")||this._symbol[p%this._symbol.length],d?(this.selectedMap[n]=d.isSelected(n),this._sIndex2ColorMap[e[p]]=d.getColor(n),a=d.getItemShape(n),a&&(this.type==r.CHART_TYPE_LINE?(a.style.iconType="legendLineIcon",a.style.symbol=this._sIndex2ShapeMap[e[p]]):t.itemStyle.normal.barBorderWidth>0&&(a.style.x+=1,a.style.y+=1,a.style.width-=2,a.style.height-=2,a.style.strokeColor=a.highlightStyle.strokeColor=t.itemStyle.normal.barBorderColor,a.highlightStyle.lineWidth=3,a.style.brushType="both"),d.setItemShape(n,a))):(this.selectedMap[n]=!0,this._sIndex2ColorMap[e[p]]=this.zr.getColor(e[p])),this.selectedMap[n]&&(i=t.stack||h+e[p],null==l[i]?(l[i]=s,m[s]=[e[p]],s++):m[l[i]].push(e[p])),c=Math.max(c,t.data.length);return{locationMap:m,maxDataLength:c}},_calculMarkMapXY:function(e,t,i){for(var n=this.series,a=0,o=t.length;o>a;a++)for(var r=0,s=t[a].length;s>r;r++){var l=t[a][r],h="xy"==i?0:"";if("-1"!=i.indexOf("x")){e[l]["counter"+h]>0&&(e[l]["average"+h]=(e[l]["sum"+h]/e[l]["counter"+h]).toFixed(2)-0);var d=this.component.xAxis.getAxis(n[l].xAxisIndex||0).getCoord(e[l]["average"+h]);e[l]["averageLine"+h]=[[d,this.component.grid.getYend()],[d,this.component.grid.getY()]],e[l]["minLine"+h]=[[e[l]["minX"+h],this.component.grid.getYend()],[e[l]["minX"+h],this.component.grid.getY()]],e[l]["maxLine"+h]=[[e[l]["maxX"+h],this.component.grid.getYend()],[e[l]["maxX"+h],this.component.grid.getY()]],e[l].isHorizontal=!1}if(h="xy"==i?1:"","-1"!=i.indexOf("y")){e[l]["counter"+h]>0&&(e[l]["average"+h]=(e[l]["sum"+h]/e[l]["counter"+h]).toFixed(2)-0);var m=this.component.yAxis.getAxis(n[l].yAxisIndex||0).getCoord(e[l]["average"+h]);e[l]["averageLine"+h]=[[this.component.grid.getX(),m],[this.component.grid.getXend(),m]],e[l]["minLine"+h]=[[this.component.grid.getX(),e[l]["minY"+h]],[this.component.grid.getXend(),e[l]["minY"+h]]],e[l]["maxLine"+h]=[[this.component.grid.getX(),e[l]["maxY"+h]],[this.component.grid.getXend(),e[l]["maxY"+h]]],e[l].isHorizontal=!0}}},addLabel:function(e,t,i,n,a){var o=[i,t],r=this.deepMerge(o,"itemStyle.normal.label"),s=this.deepMerge(o,"itemStyle.emphasis.label"),l=r.textStyle||{},h=s.textStyle||{};return r.show&&(e.style.text=this._getLabelText(t,i,n,"normal"),e.style.textPosition=null==r.position?"horizontal"===a?"right":"top":r.position,e.style.textColor=l.color,e.style.textFont=this.getFont(l)),s.show&&(e.highlightStyle.text=this._getLabelText(t,i,n,"emphasis"),e.highlightStyle.textPosition=r.show?e.style.textPosition:null==s.position?"horizontal"===a?"right":"top":s.position,e.highlightStyle.textColor=h.color,e.highlightStyle.textFont=this.getFont(h)),e},_getLabelText:function(e,t,i,n){var a=this.deepQuery([t,e],"itemStyle."+n+".label.formatter");a||"emphasis"!==n||(a=this.deepQuery([t,e],"itemStyle.normal.label.formatter"));var o=null!=t?null!=t.value?t.value:t:"-";return a?"function"==typeof a?a.call(this.myChart,e.name,i,o):"string"==typeof a?(a=a.replace("{a}","{a0}").replace("{b}","{b0}").replace("{c}","{c0}"),a=a.replace("{a0}",e.name).replace("{b0}",i).replace("{c0}",this.numAddCommas(o))):void 0:this.numAddCommas(o)},buildMark:function(e){var t=this.series[e];this.selectedMap[t.name]&&(t.markPoint&&this._buildMarkPoint(e),t.markLine&&this._buildMarkLine(e))},_buildMarkPoint:function(e){for(var t,i,n=(this.markAttachStyle||{})[e],a=this.series[e],o=this.getZlevelBase(),s=m.clone(a.markPoint),l=0,h=s.data.length;h>l;l++)t=s.data[l],i=this.getMarkCoord(e,t),s.data[l].x=null!=t.x?t.x:i[0],s.data[l].y=null!=t.y?t.y:i[1],!t.type||"max"!==t.type&&"min"!==t.type||(s.data[l].value=i[3],s.data[l].name=t.name||t.type,s.data[l].symbolSize=s.data[l].symbolSize||c.getTextWidth(i[3],this.getFont())/2+5);for(var d=this._markPoint(e,s),l=0,h=d.length;h>l;l++){d[l].zlevel=o+1;for(var p in n)d[l][p]=m.clone(n[p]);this.shapeList.push(d[l])}if(this.type===r.CHART_TYPE_FORCE||this.type===r.CHART_TYPE_CHORD)for(var l=0,h=d.length;h>l;l++)this.zr.addShape(d[l])},_buildMarkLine:function(e){for(var t,i,n=(this.markAttachStyle||{})[e],a=this.series[e],o=this.getZlevelBase(),s=m.clone(a.markLine),l=0,h=s.data.length;h>l;l++)t=s.data[l],!t.type||"max"!==t.type&&"min"!==t.type&&"average"!==t.type?i=[this.getMarkCoord(e,t[0]),this.getMarkCoord(e,t[1])]:(i=this.getMarkCoord(e,t),s.data[l]=[m.clone(t),{}],s.data[l][0].name=t.name||t.type,s.data[l][0].value=i[3],i=i[2],t=[{},{}]),null!=i&&null!=i[0]&&null!=i[1]&&(s.data[l][0].x=null!=t[0].x?t[0].x:i[0][0],s.data[l][0].y=null!=t[0].y?t[0].y:i[0][1],s.data[l][1].x=null!=t[1].x?t[1].x:i[1][0],s.data[l][1].y=null!=t[1].y?t[1].y:i[1][1]);for(var d=this._markLine(e,s),l=0,h=d.length;h>l;l++){d[l].zlevel=o+1;for(var c in n)d[l][c]=m.clone(n[c]);this.shapeList.push(d[l])}if(this.type===r.CHART_TYPE_FORCE||this.type===r.CHART_TYPE_CHORD)for(var l=0,h=d.length;h>l;l++)this.zr.addShape(d[l])},_markPoint:function(e,t){var i=this.series[e],n=this.component;m.merge(t,this.ecTheme.markPoint),t.name=i.name;var a,o,l,h,d,c,p,u=[],V=t.data,U=n.dataRange,y=n.legend,g=this.zr.getWidth(),f=this.zr.getHeight();if(t.large)a=this.getLargeMarkPoingShape(e,t),a._mark="largePoint",a&&u.push(a);else for(var b=0,_=V.length;_>b;b++)null!=V[b].x&&null!=V[b].y&&(l=null!=V[b]&&null!=V[b].value?V[b].value:"",y&&(o=y.getColor(i.name)),U&&(o=isNaN(l)?o:U.getColor(l),h=[V[b],t],d=this.deepQuery(h,"itemStyle.normal.color")||o,c=this.deepQuery(h,"itemStyle.emphasis.color")||d,null==d&&null==c)||(o=null==o?this.zr.getColor(e):o,V[b].tooltip=V[b].tooltip||t.tooltip||{trigger:"item"},V[b].name=null!=V[b].name?V[b].name:"",V[b].value=l,a=this.getSymbolShape(t,e,V[b],b,V[b].name,this.parsePercent(V[b].x,g),this.parsePercent(V[b].y,f),"pin",o,"rgba(0,0,0,0)","horizontal"),a._mark="point",p=this.deepMerge([V[b],t],"effect"),p.show&&(a.effect=p),i.type===r.CHART_TYPE_MAP&&(a._geo=this.getMarkGeo(V[b])),s.pack(a,i,e,V[b],b,V[b].name,l),u.push(a)));return u},_markLine:function(e,t){var i=this.series[e],n=this.component;m.merge(t,this.ecTheme.markLine),t.symbol=t.symbol instanceof Array?t.symbol.length>1?t.symbol:[t.symbol[0],t.symbol[0]]:[t.symbol,t.symbol],t.symbolSize=t.symbolSize instanceof Array?t.symbolSize.length>1?t.symbolSize:[t.symbolSize[0],t.symbolSize[0]]:[t.symbolSize,t.symbolSize],t.symbolRotate=t.symbolRotate instanceof Array?t.symbolRotate.length>1?t.symbolRotate:[t.symbolRotate[0],t.symbolRotate[0]]:[t.symbolRotate,t.symbolRotate],t.name=i.name;for(var a,o,l,h,d,c,p,u,V=[],U=t.data,y=n.dataRange,g=n.legend,f=this.zr.getWidth(),b=this.zr.getHeight(),_=0,x=U.length;x>_;_++)null!=U[_][0].x&&null!=U[_][0].y&&null!=U[_][1].x&&null!=U[_][1].y&&(o=g?g.getColor(i.name):this.zr.getColor(e),u=this.deepMerge(U[_]),l=null!=u&&null!=u.value?u.value:"",y&&(o=isNaN(l)?o:y.getColor(l),h=[u,t],d=this.deepQuery(h,"itemStyle.normal.color")||o,c=this.deepQuery(h,"itemStyle.emphasis.color")||d,null==d&&null==c)||(U[_][0].tooltip=u.tooltip||{trigger:"item"},U[_][0].name=null!=U[_][0].name?U[_][0].name:"",U[_][1].name=null!=U[_][1].name?U[_][1].name:"",U[_][0].value=null!=U[_][0].value?U[_][0].value:"",a=this.getLineMarkShape(t,e,U[_],_,this.parsePercent(U[_][0].x,f),this.parsePercent(U[_][0].y,b),this.parsePercent(U[_][1].x,f),this.parsePercent(U[_][1].y,b),o),a._mark="line",p=this.deepMerge([u,t],"effect"),p.show&&(a.effect=p),i.type===r.CHART_TYPE_MAP&&(a._geo=[this.getMarkGeo(U[_][0]),this.getMarkGeo(U[_][1])]),s.pack(a,i,e,U[_][0],_,U[_][0].name+(""!==U[_][1].name?" > "+U[_][1].name:""),l),V.push(a)));return V},getMarkCoord:function(){return[0,0]},getSymbolShape:function(e,t,a,o,r,l,h,d,m,c,p){var u=[a,e],V=null!=a?null!=a.value?a.value:a:"-";d=this.deepQuery(u,"symbol")||d;var U=this.deepQuery(u,"symbolSize");U="function"==typeof U?U(V):U;var y=this.deepQuery(u,"symbolRotate"),g=this.deepMerge(u,"itemStyle.normal"),f=this.deepMerge(u,"itemStyle.emphasis"),b=null!=g.borderWidth?g.borderWidth:g.lineStyle&&g.lineStyle.width;null==b&&(b=d.match("empty")?2:0);var _=null!=f.borderWidth?f.borderWidth:f.lineStyle&&f.lineStyle.width;null==_&&(_=b+2);var x=new n({style:{iconType:d.replace("empty","").toLowerCase(),x:l-U,y:h-U,width:2*U,height:2*U,brushType:"both",color:d.match("empty")?c:this.getItemStyleColor(g.color,t,o,a)||m,strokeColor:g.borderColor||this.getItemStyleColor(g.color,t,o,a)||m,lineWidth:b},highlightStyle:{color:d.match("empty")?c:this.getItemStyleColor(f.color,t,o,a),strokeColor:f.borderColor||g.borderColor||this.getItemStyleColor(g.color,t,o,a)||m,lineWidth:_},clickable:this.deepQuery(u,"clickable")});return d.match("image")&&(x.style.image=d.replace(new RegExp("^image:\\/\\/"),""),x=new i({style:x.style,highlightStyle:x.highlightStyle,clickable:this.deepQuery(u,"clickable")})),null!=y&&(x.rotation=[y*Math.PI/180,l,h]),d.match("star")&&(x.style.iconType="star",x.style.n=d.replace("empty","").replace("star","")-0||5),"none"===d&&(x.invisible=!0,x.hoverable=!1),x=this.addLabel(x,e,a,r,p),d.match("empty")&&(null==x.style.textColor&&(x.style.textColor=x.style.strokeColor),null==x.highlightStyle.textColor&&(x.highlightStyle.textColor=x.highlightStyle.strokeColor)),s.pack(x,e,t,a,o,r),x._x=l,x._y=h,x._dataIndex=o,x._seriesIndex=t,x},getLineMarkShape:function(e,t,i,n,o,r,s,l,h){var d=null!=i[0]?null!=i[0].value?i[0].value:i[0]:"-",m=null!=i[1]?null!=i[1].value?i[1].value:i[1]:"-",c=[this.query(i[0],"symbol")||e.symbol[0],this.query(i[1],"symbol")||e.symbol[1]],p=[this.query(i[0],"symbolSize")||e.symbolSize[0],this.query(i[1],"symbolSize")||e.symbolSize[1]];p[0]="function"==typeof p[0]?p[0](d):p[0],p[1]="function"==typeof p[1]?p[1](m):p[1];var u=[this.query(i[0],"symbolRotate")||e.symbolRotate[0],this.query(i[1],"symbolRotate")||e.symbolRotate[1]],V=[i[0],e],U=this.deepMerge(V,"itemStyle.normal");U.color=this.getItemStyleColor(U.color,t,n,i);var y=this.deepMerge(V,"itemStyle.emphasis");y.color=this.getItemStyleColor(y.color,t,n,i);var g=U.lineStyle,f=y.lineStyle,b=g.width;null==b&&(b=U.borderWidth);var _=f.width;null==_&&(_=null!=y.borderWidth?y.borderWidth:b+2);var x=new a({style:{smooth:e.smooth?"spline":!1,symbol:c,symbolSize:p,symbolRotate:u,xStart:o,yStart:r,xEnd:s,yEnd:l,brushType:"both",lineType:g.type,shadowColor:g.shadowColor||g.color||U.borderColor||U.color||h,shadowBlur:g.shadowBlur,shadowOffsetX:g.shadowOffsetX,shadowOffsetY:g.shadowOffsetY,color:U.color||h,strokeColor:g.color||U.borderColor||U.color||h,lineWidth:b,symbolBorderColor:U.borderColor||U.color||h,symbolBorder:U.borderWidth},highlightStyle:{shadowColor:f.shadowColor,shadowBlur:f.shadowBlur,shadowOffsetX:f.shadowOffsetX,shadowOffsetY:f.shadowOffsetY,color:y.color||U.color||h,strokeColor:f.color||g.color||y.borderColor||U.borderColor||y.color||U.color||h,lineWidth:_,symbolBorderColor:y.borderColor||U.borderColor||y.color||U.color||h,symbolBorder:null==y.borderWidth?U.borderWidth+2:y.borderWidth},clickable:this.deepQuery(V,"clickable")});return x=this.addLabel(x,e,i[0],i[0].name+" : "+i[1].name),x._x=s,x._y=l,x},getLargeMarkPoingShape:function(e,t){var i,n,a,r,s,l,h=this.series[e],d=this.component,m=t.data,c=d.dataRange,p=d.legend,u=[m[0],t];if(p&&(n=p.getColor(h.name)),!c||(a=null!=m[0]?null!=m[0].value?m[0].value:m[0]:"-",n=isNaN(a)?n:c.getColor(a),r=this.deepQuery(u,"itemStyle.normal.color")||n,s=this.deepQuery(u,"itemStyle.emphasis.color")||r,null!=r||null!=s)){n=this.deepMerge(u,"itemStyle.normal").color||n;var V=this.deepQuery(u,"symbol")||"circle";V=V.replace("empty","").replace(/\d/g,""),l=this.deepMerge([m[0],t],"effect");var U=window.devicePixelRatio||1;return i=new o({style:{pointList:m,color:n,strokeColor:n,shadowColor:l.shadowColor||n,shadowBlur:(null!=l.shadowBlur?l.shadowBlur:8)*U,size:this.deepQuery(u,"symbolSize"),iconType:V,brushType:"fill",lineWidth:1},draggable:!1,hoverable:!1}),l.show&&(i.effect=l),i}},backupShapeList:function(){this.shapeList&&this.shapeList.length>0?(this.lastShapeList=this.shapeList,this.shapeList=[]):this.lastShapeList=[]},addShapeList:function(){var e,t,i=this.option.animationThreshold/(this.canvasSupported?2:4),n=this.lastShapeList,a=this.shapeList,o=n.length>0?500:this.query(this.option,"animationDuration"),s=this.query(this.option,"animationEasing"),l={},h={};if(this.option.animation&&!this.option.renderAsImage&&a.lengthd;d++)t=this._getAnimationKey(n[d]),t.match("undefined")?this.zr.delShape(n[d].id):(t+=n[d].type,l[t]=n[d]);for(var d=0,m=a.length;m>d;d++)t=this._getAnimationKey(a[d]),t.match("undefined")?this.zr.addShape(a[d]):(t+=a[d].type,h[t]=a[d]);for(t in l)h[t]||this.zr.delShape(l[t].id);for(t in h)l[t]?(this.zr.delShape(l[t].id),this._animateMod(l[t],h[t],o,s)):(e=this.type!=r.CHART_TYPE_LINE&&this.type!=r.CHART_TYPE_RADAR||0===t.indexOf("icon")?0:o/2,this._animateMod(!1,h[t],o,s,e));this.zr.refresh(),this.animationEffect()}else{this.motionlessOnce=!1,this.zr.delShape(n);for(var d=0,m=a.length;m>d;d++)this.zr.addShape(a[d])}},_getAnimationKey:function(e){return this.type!=r.CHART_TYPE_MAP?s.get(e,"seriesIndex")+"_"+s.get(e,"dataIndex")+(e._mark?e._mark:"")+(this.type===r.CHART_TYPE_RADAR?s.get(e,"special"):""):s.get(e,"seriesIndex")+"_"+s.get(e,"dataIndex")+(e._mark?e._mark:"undefined")},_animateMod:function(e,t,i,n,a){switch(t.type){case"broken-line":case"half-smooth-polygon":l.pointList(this.zr,e,t,i,n);break;case"rectangle":l.rectangle(this.zr,e,t,i,n);break;case"icon":l.icon(this.zr,e,t,i,n,a);break;case"candle":i>500?l.candle(this.zr,e,t,i,n):this.zr.addShape(t);break;case"ring":case"sector":case"circle":i>500?l.ring(this.zr,e,t,i+(s.get(t,"dataIndex")||0)%20*100,n):"sector"===t.type?l.sector(this.zr,e,t,i,n):this.zr.addShape(t);break;case"text":l.text(this.zr,e,t,i,n);break;case"polygon":i>500?l.polygon(this.zr,e,t,i,n):l.pointList(this.zr,e,t,i,n);break;case"ribbon":l.ribbon(this.zr,e,t,i,n);break;case"gauge-pointer":l.gaugePointer(this.zr,e,t,i,n);break;case"mark-line":l.markline(this.zr,e,t,i,n);break;case"bezier-curve":case"line":l.line(this.zr,e,t,i,n);break;default:this.zr.addShape(t)}},animationMark:function(e,t,i){for(var n=i||this.shapeList,a=0,o=n.length;o>a;a++)n[a]._mark&&this._animateMod(!1,n[a],e,t);this.animationEffect(i)},animationEffect:function(e){!e&&this.clearEffectShape();var t=e||this.shapeList;if(null!=t){var i=r.EFFECT_ZLEVEL;this.canvasSupported&&this.zr.modLayer(i,{motionBlur:!0,lastFrameAlpha:.95});for(var n,a=0,o=t.length;o>a;a++)n=t[a],n._mark&&n.effect&&n.effect.show&&h[n._mark]&&(h[n._mark](this.zr,this.effectList,n,i),this.effectList[this.effectList.length-1]._mark=n._mark)}},clearEffectShape:function(e){this.zr&&this.effectList&&this.effectList.length>0&&(e&&this.zr.modLayer(r.EFFECT_ZLEVEL,{motionBlur:!1}),this.zr.delShape(this.effectList)),this.effectList=[]},addMark:function(e,t,i){var n=this.series[e];if(this.selectedMap[n.name]){var a=500,o=this.query(this.option,"animationEasing"),r=n[i].data,s=this.shapeList.length;if(n[i].data=t.data,this["_build"+i.replace("m","M")](e),this.option.animation&&!this.option.renderAsImage)this.animationMark(a,o,this.shapeList.slice(s));else{for(var l=s,h=this.shapeList.length;h>l;l++)this.zr.addShape(this.shapeList[l]);this.zr.refresh()}n[i].data=r}},delMark:function(e,t,i){i=i.replace("mark","").replace("large","").toLowerCase();var n=this.series[e];if(this.selectedMap[n.name]){for(var a=!1,o=[this.shapeList,this.effectList],r=2;r--;)for(var l=0,h=o[r].length;h>l;l++)if(o[r][l]._mark==i&&s.get(o[r][l],"seriesIndex")==e&&s.get(o[r][l],"name")==t){this.zr.delShape(o[r][l].id),o[r].splice(l,1),a=!0;break}a&&this.zr.refresh()}}},t}),define("zrender/shape/Circle",["require","./Base","../tool/util"],function(e){"use strict";var t=e("./Base"),i=function(e){t.call(this,e)};return i.prototype={type:"circle",buildPath:function(e,t){e.arc(t.x,t.y,t.r,0,2*Math.PI,!0)},getRect:function(e){if(e.__rect)return e.__rect;var t;return t="stroke"==e.brushType||"fill"==e.brushType?e.lineWidth||1:0,e.__rect={x:Math.round(e.x-e.r-t/2),y:Math.round(e.y-e.r-t/2),width:2*e.r+t,height:2*e.r+t},e.__rect}},e("../tool/util").inherits(i,t),i}),define("echarts/util/accMath",[],function(){function e(e,t){var i=e.toString(),n=t.toString(),a=0;try{a=n.split(".")[1].length}catch(o){}try{a-=i.split(".")[1].length}catch(o){}return(i.replace(".","")-0)/(n.replace(".","")-0)*Math.pow(10,a)}function t(e,t){var i=e.toString(),n=t.toString(),a=0;try{a+=i.split(".")[1].length}catch(o){}try{a+=n.split(".")[1].length}catch(o){}return(i.replace(".","")-0)*(n.replace(".","")-0)/Math.pow(10,a)}function i(e,t){var i=0,n=0;try{i=e.toString().split(".")[1].length}catch(a){}try{n=t.toString().split(".")[1].length}catch(a){}var o=Math.pow(10,Math.max(i,n));return(Math.round(e*o)+Math.round(t*o))/o}function n(e,t){return i(e,-t)}return{accDiv:e,accMul:t,accAdd:i,accSub:n}}),define("echarts/util/ecQuery",["require","zrender/tool/util"],function(e){function t(e,t){if("undefined"!=typeof e){if(!t)return e;t=t.split(".");for(var i=t.length,n=0;i>n;){if(e=e[t[n]],"undefined"==typeof e)return;n++}return e}}function i(e,i){for(var n,a=0,o=e.length;o>a;a++)if(n=t(e[a],i),"undefined"!=typeof n)return n}function n(e,i){for(var n,o=e.length;o--;){var r=t(e[o],i);"undefined"!=typeof r&&("undefined"==typeof n?n=a.clone(r):a.merge(n,r,!0))}return n}var a=e("zrender/tool/util");return{query:t,deepQuery:i,deepMerge:n}}),define("echarts/util/number",[],function(){function e(e){return e.replace(/^\s+/,"").replace(/\s+$/,"")}function t(t,i){return"string"==typeof t?e(t).match(/%$/)?parseFloat(t)/100*i:parseFloat(t):t}function i(e,i){return[t(i[0],e.getWidth()),t(i[1],e.getHeight())]}function n(e,i){i instanceof Array||(i=[0,i]);var n=Math.min(e.getWidth(),e.getHeight())/2;return[t(i[0],n),t(i[1],n)]}function a(e){return isNaN(e)?"-":(e=(e+"").split("."),e[0].replace(/(\d{1,3})(?=(?:\d{3})+(?!\d))/g,"$1,")+(e.length>1?"."+e[1]:""))}return{parsePercent:t,parseCenter:i,parseRadius:n,addCommas:a}}),define("echarts/util/shape/Icon",["require","zrender/tool/util","zrender/shape/Star","zrender/shape/Heart","zrender/shape/Droplet","zrender/shape/Image","zrender/shape/Base"],function(e){function t(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;e.moveTo(i,n+t.height),e.lineTo(i+5*a,n+14*o),e.lineTo(i+t.width,n+3*o),e.lineTo(i+13*a,n),e.lineTo(i+2*a,n+11*o),e.lineTo(i,n+t.height),e.moveTo(i+6*a,n+10*o),e.lineTo(i+14*a,n+2*o),e.moveTo(i+10*a,n+13*o),e.lineTo(i+t.width,n+13*o),e.moveTo(i+13*a,n+10*o),e.lineTo(i+13*a,n+t.height)}function i(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;e.moveTo(i,n+t.height),e.lineTo(i+5*a,n+14*o),e.lineTo(i+t.width,n+3*o),e.lineTo(i+13*a,n),e.lineTo(i+2*a,n+11*o),e.lineTo(i,n+t.height),e.moveTo(i+6*a,n+10*o),e.lineTo(i+14*a,n+2*o),e.moveTo(i+10*a,n+13*o),e.lineTo(i+t.width,n+13*o)}function n(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;e.moveTo(i+4*a,n+15*o),e.lineTo(i+9*a,n+13*o),e.lineTo(i+14*a,n+8*o),e.lineTo(i+11*a,n+5*o),e.lineTo(i+6*a,n+10*o),e.lineTo(i+4*a,n+15*o),e.moveTo(i+5*a,n),e.lineTo(i+11*a,n),e.moveTo(i+5*a,n+o),e.lineTo(i+11*a,n+o),e.moveTo(i,n+2*o),e.lineTo(i+t.width,n+2*o),e.moveTo(i,n+5*o),e.lineTo(i+3*a,n+t.height),e.lineTo(i+13*a,n+t.height),e.lineTo(i+t.width,n+5*o)}function a(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;e.moveTo(i,n+3*o),e.lineTo(i+6*a,n+3*o),e.moveTo(i+3*a,n),e.lineTo(i+3*a,n+6*o),e.moveTo(i+3*a,n+8*o),e.lineTo(i+3*a,n+t.height),e.lineTo(i+t.width,n+t.height),e.lineTo(i+t.width,n+3*o),e.lineTo(i+8*a,n+3*o)}function o(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;e.moveTo(i+6*a,n),e.lineTo(i+2*a,n+3*o),e.lineTo(i+6*a,n+6*o),e.moveTo(i+2*a,n+3*o),e.lineTo(i+14*a,n+3*o),e.lineTo(i+14*a,n+11*o),e.moveTo(i+2*a,n+5*o),e.lineTo(i+2*a,n+13*o),e.lineTo(i+14*a,n+13*o),e.moveTo(i+10*a,n+10*o),e.lineTo(i+14*a,n+13*o),e.lineTo(i+10*a,n+t.height)}function r(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16,r=t.width/2;e.lineWidth=1.5,e.arc(i+r,n+r,r-a,0,2*Math.PI/3),e.moveTo(i+3*a,n+t.height),e.lineTo(i+0*a,n+12*o),e.lineTo(i+5*a,n+11*o),e.moveTo(i,n+8*o),e.arc(i+r,n+r,r-a,Math.PI,5*Math.PI/3),e.moveTo(i+13*a,n),e.lineTo(i+t.width,n+4*o),e.lineTo(i+11*a,n+5*o)}function s(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;e.moveTo(i,n),e.lineTo(i,n+t.height),e.lineTo(i+t.width,n+t.height),e.moveTo(i+2*a,n+14*o),e.lineTo(i+7*a,n+6*o),e.lineTo(i+11*a,n+11*o),e.lineTo(i+15*a,n+2*o)}function l(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;e.moveTo(i,n),e.lineTo(i,n+t.height),e.lineTo(i+t.width,n+t.height),e.moveTo(i+3*a,n+14*o),e.lineTo(i+3*a,n+6*o),e.lineTo(i+4*a,n+6*o),e.lineTo(i+4*a,n+14*o),e.moveTo(i+7*a,n+14*o),e.lineTo(i+7*a,n+2*o),e.lineTo(i+8*a,n+2*o),e.lineTo(i+8*a,n+14*o),e.moveTo(i+11*a,n+14*o),e.lineTo(i+11*a,n+9*o),e.lineTo(i+12*a,n+9*o),e.lineTo(i+12*a,n+14*o)}function h(e,t){var i=t.x,n=t.y,a=t.width-2,o=t.height-2,r=Math.min(a,o)/2;n+=2,e.moveTo(i+r+3,n+r-3),e.arc(i+r+3,n+r-3,r-1,0,-Math.PI/2,!0),e.lineTo(i+r+3,n+r-3),e.moveTo(i+r,n),e.lineTo(i+r,n+r),e.arc(i+r,n+r,r,-Math.PI/2,2*Math.PI,!0),e.lineTo(i+r,n+r),e.lineWidth=1.5}function d(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;n-=o,e.moveTo(i+1*a,n+2*o),e.lineTo(i+15*a,n+2*o),e.lineTo(i+14*a,n+3*o),e.lineTo(i+2*a,n+3*o),e.moveTo(i+3*a,n+6*o),e.lineTo(i+13*a,n+6*o),e.lineTo(i+12*a,n+7*o),e.lineTo(i+4*a,n+7*o),e.moveTo(i+5*a,n+10*o),e.lineTo(i+11*a,n+10*o),e.lineTo(i+10*a,n+11*o),e.lineTo(i+6*a,n+11*o),e.moveTo(i+7*a,n+14*o),e.lineTo(i+9*a,n+14*o),e.lineTo(i+8*a,n+15*o),e.lineTo(i+7*a,n+15*o)}function m(e,t){var i=t.x,n=t.y,a=t.width,o=t.height,r=a/16,s=o/16,l=2*Math.min(r,s);e.moveTo(i+r+l,n+s+l),e.arc(i+r,n+s,l,Math.PI/4,3*Math.PI),e.lineTo(i+7*r-l,n+6*s-l),e.arc(i+7*r,n+6*s,l,Math.PI/4*5,4*Math.PI),e.arc(i+7*r,n+6*s,l/2,Math.PI/4*5,4*Math.PI),e.moveTo(i+7*r-l/2,n+6*s+l),e.lineTo(i+r+l,n+14*s-l),e.arc(i+r,n+14*s,l,-Math.PI/4,2*Math.PI),e.moveTo(i+7*r+l/2,n+6*s),e.lineTo(i+14*r-l,n+10*s-l/2),e.moveTo(i+16*r,n+10*s),e.arc(i+14*r,n+10*s,l,0,3*Math.PI),e.lineWidth=1.5}function c(e,t){var i=t.x,n=t.y,a=t.width,o=t.height,r=Math.min(a,o)/2;e.moveTo(i+a,n+o/2),e.arc(i+r,n+r,r,0,2*Math.PI),e.arc(i+r,n,r,Math.PI/4,Math.PI/5*4),e.arc(i,n+r,r,-Math.PI/3,Math.PI/3),e.arc(i+a,n+o,r,Math.PI,Math.PI/2*3),e.lineWidth=1.5}function p(e,t){for(var i=t.x,n=t.y,a=t.width,o=t.height,r=Math.round(o/3),s=3;s--;)e.rect(i,n+r*s+2,a,2)}function u(e,t){for(var i=t.x,n=t.y,a=t.width,o=t.height,r=Math.round(a/3),s=3;s--;)e.rect(i+r*s,n,2,o)}function V(e,t){var i=t.x,n=t.y,a=t.width/16;e.moveTo(i+a,n),e.lineTo(i+a,n+t.height),e.lineTo(i+15*a,n+t.height),e.lineTo(i+15*a,n),e.lineTo(i+a,n),e.moveTo(i+3*a,n+3*a),e.lineTo(i+13*a,n+3*a),e.moveTo(i+3*a,n+6*a),e.lineTo(i+13*a,n+6*a),e.moveTo(i+3*a,n+9*a),e.lineTo(i+13*a,n+9*a),e.moveTo(i+3*a,n+12*a),e.lineTo(i+9*a,n+12*a)}function U(e,t){var i=t.x,n=t.y,a=t.width/16,o=t.height/16;e.moveTo(i,n),e.lineTo(i,n+t.height),e.lineTo(i+t.width,n+t.height),e.lineTo(i+t.width,n),e.lineTo(i,n),e.moveTo(i+4*a,n),e.lineTo(i+4*a,n+8*o),e.lineTo(i+12*a,n+8*o),e.lineTo(i+12*a,n),e.moveTo(i+6*a,n+11*o),e.lineTo(i+6*a,n+13*o),e.lineTo(i+10*a,n+13*o),e.lineTo(i+10*a,n+11*o),e.lineTo(i+6*a,n+11*o)}function y(e,t){var i=t.x,n=t.y,a=t.width,o=t.height;e.moveTo(i,n+o/2),e.lineTo(i+a,n+o/2),e.moveTo(i+a/2,n),e.lineTo(i+a/2,n+o)}function g(e,t){var i=t.width/2,n=t.height/2,a=Math.min(i,n);e.moveTo(t.x+i+a,t.y+n),e.arc(t.x+i,t.y+n,a,0,2*Math.PI),e.closePath()}function f(e,t){e.rect(t.x,t.y,t.width,t.height),e.closePath()}function b(e,t){var i=t.width/2,n=t.height/2,a=t.x+i,o=t.y+n,r=Math.min(i,n);e.moveTo(a,o-r),e.lineTo(a+r,o+r),e.lineTo(a-r,o+r),e.lineTo(a,o-r),e.closePath()}function _(e,t){var i=t.width/2,n=t.height/2,a=t.x+i,o=t.y+n,r=Math.min(i,n);e.moveTo(a,o-r),e.lineTo(a+r,o),e.lineTo(a,o+r),e.lineTo(a-r,o),e.lineTo(a,o-r),e.closePath()}function x(e,t){var i=t.x,n=t.y,a=t.width/16;e.moveTo(i+8*a,n),e.lineTo(i+a,n+t.height),e.lineTo(i+8*a,n+t.height/4*3),e.lineTo(i+15*a,n+t.height),e.lineTo(i+8*a,n),e.closePath()}function k(t,i){var n=e("zrender/shape/Star"),a=i.width/2,o=i.height/2;n.prototype.buildPath(t,{x:i.x+a,y:i.y+o,r:Math.min(a,o),n:i.n||5})}function L(t,i){var n=e("zrender/shape/Heart");n.prototype.buildPath(t,{x:i.x+i.width/2,y:i.y+.2*i.height,a:i.width/2,b:.8*i.height})}function v(t,i){var n=e("zrender/shape/Droplet");n.prototype.buildPath(t,{x:i.x+.5*i.width,y:i.y+.5*i.height,a:.5*i.width,b:.8*i.height})}function W(e,t){var i=t.x,n=t.y-t.height/2*1.5,a=t.width/2,o=t.height/2,r=Math.min(a,o);e.arc(i+a,n+o,r,Math.PI/5*4,Math.PI/5),e.lineTo(i+a,n+o+1.5*r),e.closePath()}function w(t,i,n){var a=e("zrender/shape/Image");this._imageShape=this._imageShape||new a({style:{}});for(var o in i)this._imageShape.style[o]=i[o];this._imageShape.brush(t,!1,n)}function X(e){K.call(this,e)}var I=e("zrender/tool/util"),K=e("zrender/shape/Base");return X.prototype={type:"icon",iconLibrary:{mark:t,markUndo:i,markClear:n,dataZoom:a,dataZoomReset:o,restore:r,lineChart:s,barChart:l,pieChart:h,funnelChart:d,forceChart:m,chordChart:c,stackChart:p,tiledChart:u,dataView:V,saveAsImage:U,cross:y,circle:g,rectangle:f,triangle:b,diamond:_,arrow:x,star:k,heart:L,droplet:v,pin:W,image:w},brush:function(t,i,n){var a=i?this.highlightStyle:this.style;a=a||{};var o=a.iconType||this.style.iconType;if("image"===o){var r=e("zrender/shape/Image");r.prototype.brush.call(this,t,i,n)}else{var a=this.beforeBrush(t,i);switch(t.beginPath(),this.buildPath(t,a,n),a.brushType){case"both":t.fill();case"stroke":a.lineWidth>0&&t.stroke();break;default:t.fill()}this.drawText(t,a,this.style),this.afterBrush(t)}},buildPath:function(e,t,i){this.iconLibrary[t.iconType]?this.iconLibrary[t.iconType].call(this,e,t,i):(e.moveTo(t.x,t.y),e.lineTo(t.x+t.width,t.y),e.lineTo(t.x+t.width,t.y+t.height),e.lineTo(t.x,t.y+t.height),e.lineTo(t.x,t.y),e.closePath())},getRect:function(e){return e.__rect?e.__rect:(e.__rect={x:Math.round(e.x),y:Math.round(e.y-("pin"==e.iconType?e.height/2*1.5:0)),width:e.width,height:e.height},e.__rect)},isCover:function(e,t){var i=this.getTansform(e,t);e=i[0],t=i[1];var n=this.style.__rect;n||(n=this.style.__rect=this.getRect(this.style));var a=n.height<8||n.width<8?4:0;return e>=n.x-a&&e<=n.x+n.width+a&&t>=n.y-a&&t<=n.y+n.height+a?!0:!1}},I.inherits(X,K),X}),define("echarts/util/shape/MarkLine",["require","zrender/shape/Base","./Icon","zrender/shape/Line","zrender/shape/BrokenLine","zrender/tool/matrix","zrender/tool/area","zrender/shape/util/dashedLineTo","zrender/shape/util/smoothSpline","zrender/tool/util"],function(e){function t(e){i.call(this,e)}var i=e("zrender/shape/Base"),n=e("./Icon"),a=e("zrender/shape/Line"),o=new a({}),r=e("zrender/shape/BrokenLine"),s=new r({}),l=e("zrender/tool/matrix"),h=e("zrender/tool/area"),d=e("zrender/shape/util/dashedLineTo"),m=e("zrender/shape/util/smoothSpline"),c=e("zrender/tool/util");return t.prototype={type:"mark-line",brush:function(e,t){var i=this.style;t&&(i=this.getHighlightStyle(i,this.highlightStyle||{})),e.save(),this.setContext(e,i),this.setTransform(e),e.save(),e.beginPath(),this.buildLinePath(e,i,this.style.lineWidth||1),e.stroke(),e.restore(),this.brushSymbol(e,i,0),this.brushSymbol(e,i,1),this.drawText(e,i,this.style),e.restore()},buildLinePath:function(e,t,i){var n=t.pointList||this.getPointList(t);t.pointList=n;var a=Math.min(t.pointList.length,Math.round(t.pointListLength||t.pointList.length));if(t.lineType&&"solid"!=t.lineType){if("dashed"==t.lineType||"dotted"==t.lineType)if("spline"!==t.smooth){var o=i*("dashed"==t.lineType?5:1);e.moveTo(n[0][0],n[0][1]);for(var r=1;a>r;r++)d(e,n[r-1][0],n[r-1][1],n[r][0],n[r][1],o)}else for(var r=1;a>r;r+=2)e.moveTo(n[r-1][0],n[r-1][1]),e.lineTo(n[r][0],n[r][1])}else{e.moveTo(n[0][0],n[0][1]);for(var r=1;a>r;r++)e.lineTo(n[r][0],n[r][1])}},brushSymbol:function(e,t,i){if("none"!=t.symbol[i]){e.save(),e.beginPath(),e.lineWidth=t.symbolBorder,e.strokeStyle=t.symbolBorderColor,t.iconType=t.symbol[i].replace("empty","").toLowerCase(),t.symbol[i].match("empty")&&(e.fillStyle="#fff");var a,o=Math.min(t.pointList.length,Math.round(t.pointListLength||t.pointList.length)),r=0===i?t.pointList[0][0]:t.pointList[o-1][0],s=0===i?t.pointList[0][1]:t.pointList[o-1][1],h="undefined"!=typeof t.symbolRotate[i]?t.symbolRotate[i]-0:0;if(0!==h&&(a=l.create(),l.identity(a),(r||s)&&l.translate(a,a,[-r,-s]),l.rotate(a,a,h*Math.PI/180),(r||s)&&l.translate(a,a,[r,s]),e.transform.apply(e,a)),"arrow"==t.iconType&&0===h)this.buildArrawPath(e,t,i);else{var d=t.symbolSize[i];t.x=r-d,t.y=s-d,t.width=2*d,t.height=2*d,n.prototype.buildPath(e,t)}e.closePath(),e.fill(),e.stroke(),e.restore()}},buildArrawPath:function(e,t,i){var n=Math.min(t.pointList.length,Math.round(t.pointListLength||t.pointList.length)),a=2*t.symbolSize[i],o=t.pointList[0][0],r=t.pointList[n-1][0],s=t.pointList[0][1],l=t.pointList[n-1][1],h=0;"spline"===t.smooth&&(h=.2);var d=Math.atan(Math.abs((l-s)/(o-r)));0===i?r>o?l>s?d=2*Math.PI-d+h:d+=h:l>s?d+=Math.PI-h:d=Math.PI-d-h:o>r?s>l?d=2*Math.PI-d+h:d+=h:s>l?d+=Math.PI-h:d=Math.PI-d-h;var m=Math.PI/8,c=0===i?o:r,p=0===i?s:l,u=[[c+a*Math.cos(d-m),p-a*Math.sin(d-m)],[c+.6*a*Math.cos(d),p-.6*a*Math.sin(d)],[c+a*Math.cos(d+m),p-a*Math.sin(d+m)]];e.moveTo(c,p); +for(var V=0,U=u.length;U>V;V++)e.lineTo(u[V][0],u[V][1]);e.lineTo(c,p)},getPointList:function(e){var t=[[e.xStart,e.yStart],[e.xEnd,e.yEnd]];if("spline"===e.smooth){var i=t[1][0],n=t[1][1];t[3]=[i,n],t[1]=this.getOffetPoint(t[0],t[3]),t[2]=this.getOffetPoint(t[3],t[0]),t=m(t,!1),t[t.length-1]=[i,n]}return t},getOffetPoint:function(e,t){var i,n=Math.sqrt(Math.round((e[0]-t[0])*(e[0]-t[0])+(e[1]-t[1])*(e[1]-t[1])))/3,a=[e[0],e[1]],o=.2;if(e[0]!=t[0]&&e[1]!=t[1]){var r=(t[1]-e[1])/(t[0]-e[0]);i=Math.atan(r)}else i=e[0]==t[0]?(e[1]<=t[1]?1:-1)*Math.PI/2:0;var s,l;return e[0]<=t[0]?(i-=o,s=Math.round(Math.cos(i)*n),l=Math.round(Math.sin(i)*n),a[0]+=s,a[1]+=l):(i+=o,s=Math.round(Math.cos(i)*n),l=Math.round(Math.sin(i)*n),a[0]-=s,a[1]-=l),a},getRect:function(e){if(e.__rect)return e.__rect;var t=e.lineWidth||1;return e.__rect={x:Math.min(e.xStart,e.xEnd)-t,y:Math.min(e.yStart,e.yEnd)-t,width:Math.abs(e.xStart-e.xEnd)+t,height:Math.abs(e.yStart-e.yEnd)+t},e.__rect},isCover:function(e,t){var i=this.getTansform(e,t);e=i[0],t=i[1];var n=this.style.__rect;return n||(n=this.style.__rect=this.getRect(this.style)),e>=n.x&&e<=n.x+n.width&&t>=n.y&&t<=n.y+n.height?"spline"!==this.style.smooth?h.isInside(o,this.style,e,t):h.isInside(s,this.style,e,t):!1}},c.inherits(t,i),t}),define("echarts/util/shape/Symbol",["require","zrender/shape/Base","zrender/shape/Polygon","zrender/tool/util","./normalIsCover"],function(e){function t(e){i.call(this,e)}var i=e("zrender/shape/Base"),n=e("zrender/shape/Polygon"),a=new n({}),o=e("zrender/tool/util");return t.prototype={type:"symbol",buildPath:function(e,t){var i=t.pointList,n=i.length;if(0!==n)for(var a,o,r,s,l,h=1e4,d=Math.ceil(n/h),m=i[0]instanceof Array,c=t.size?t.size:2,p=c,u=c/2,V=2*Math.PI,U=0;d>U;U++){e.beginPath(),a=U*h,o=a+h,o=o>n?n:o;for(var y=a;o>y;y++)if(t.random&&(r=t["randomMap"+y%20]/100,p=c*r*r,u=p/2),m?(s=i[y][0],l=i[y][1]):(s=i[y].x,l=i[y].y),3>p)e.rect(s-u,l-u,p,p);else switch(t.iconType){case"circle":e.moveTo(s,l),e.arc(s,l,u,0,V,!0);break;case"diamond":e.moveTo(s,l-u),e.lineTo(s+u/3,l-u/3),e.lineTo(s+u,l),e.lineTo(s+u/3,l+u/3),e.lineTo(s,l+u),e.lineTo(s-u/3,l+u/3),e.lineTo(s-u,l),e.lineTo(s-u/3,l-u/3),e.lineTo(s,l-u);break;default:e.rect(s-u,l-u,p,p)}if(e.closePath(),d-1>U)switch(t.brushType){case"both":e.fill(),t.lineWidth>0&&e.stroke();break;case"stroke":t.lineWidth>0&&e.stroke();break;default:e.fill()}}},getRect:function(e){return e.__rect||a.getRect(e)},isCover:e("./normalIsCover")},o.inherits(t,i),t}),define("echarts/util/ecAnimation",["require","zrender/tool/util","zrender/shape/Polygon"],function(e){function t(e,t,i,n,a){var o,r=i.style.pointList,s=r.length;if(!t){if(o=[],"vertical"!=i._orient)for(var l=r[0][1],h=0;s>h;h++)o[h]=[r[h][0],l];else for(var d=r[0][0],h=0;s>h;h++)o[h]=[d,r[h][1]];"half-smooth-polygon"==i.type&&(o[s-1]=u.clone(r[s-1]),o[s-2]=u.clone(r[s-2])),t={style:{pointList:o}}}o=t.style.pointList;var m=o.length;i.style.pointList=m==s?o:s>m?o.concat(r.slice(m)):o.slice(0,s),e.addShape(i),e.animate(i.id,"style").when(n,{pointList:r}).start(a)}function i(e,t){for(var i=arguments.length,n=2;i>n;n++){var a=arguments[n];e.style[a]=t.style[a]}}function n(e,t,n,a,o){var r=n.style;t||(t={position:n.position,style:{x:r.x,y:"vertical"==n._orient?r.y+r.height:r.y,width:"vertical"==n._orient?r.width:0,height:"vertical"!=n._orient?r.height:0}});var s=r.x,l=r.y,h=r.width,d=r.height,m=[n.position[0],n.position[1]];i(n,t,"x","y","width","height"),n.position=t.position,e.addShape(n),(m[0]!=t.position[0]||m[1]!=t.position[1])&&e.animate(n.id,"").when(a,{position:m}).start(o),e.animate(n.id,"style").when(a,{x:s,y:l,width:h,height:d}).start(o)}function a(e,t,i,n,a){if(!t){var o=i.style.y;t={style:{y:[o[0],o[0],o[0],o[0]]}}}var r=i.style.y;i.style.y=t.style.y,e.addShape(i),e.animate(i.id,"style").when(n,{y:r}).start(a)}function o(e,t,i,n,a){var o=i.style.x,r=i.style.y,s=i.style.r0,l=i.style.r;"r"!=i._animationAdd?(i.style.r0=0,i.style.r=0,i.rotation=[2*Math.PI,o,r],e.addShape(i),e.animate(i.id,"style").when(n,{r0:s,r:l}).start(a),e.animate(i.id,"").when(Math.round(n/3*2),{rotation:[0,o,r]}).start(a)):(i.style.r0=i.style.r,e.addShape(i),e.animate(i.id,"style").when(n,{r0:s}).start(a))}function r(e,t,n,a,o){t||(t="r"!=n._animationAdd?{style:{startAngle:n.style.startAngle,endAngle:n.style.startAngle}}:{style:{r0:n.style.r}});var r=n.style.startAngle,s=n.style.endAngle;i(n,t,"startAngle","endAngle"),e.addShape(n),e.animate(n.id,"style").when(a,{startAngle:r,endAngle:s}).start(o)}function s(e,t,n,a,o){t||(t={style:{x:"left"==n.style.textAlign?n.style.x+100:n.style.x-100,y:n.style.y}});var r=n.style.x,s=n.style.y;i(n,t,"x","y"),e.addShape(n),e.animate(n.id,"style").when(a,{x:r,y:s}).start(o)}function l(t,i,n,a,o){var r=e("zrender/shape/Polygon").prototype.getRect(n.style),s=r.x+r.width/2,l=r.y+r.height/2;n.scale=[.1,.1,s,l],t.addShape(n),t.animate(n.id,"").when(a,{scale:[1,1,s,l]}).start(o)}function h(e,t,n,a,o){t||(t={style:{source0:0,source1:n.style.source1>0?360:-360,target0:0,target1:n.style.target1>0?360:-360}});var r=n.style.source0,s=n.style.source1,l=n.style.target0,h=n.style.target1;t.style&&i(n,t,"source0","source1","target0","target1"),e.addShape(n),e.animate(n.id,"style").when(a,{source0:r,source1:s,target0:l,target1:h}).start(o)}function d(e,t,i,n,a){t||(t={style:{angle:i.style.startAngle}});var o=i.style.angle;i.style.angle=t.style.angle,e.addShape(i),e.animate(i.id,"style").when(n,{angle:o}).start(a)}function m(e,t,i,a,o,r){if(i.style._x=i.style.x,i.style._y=i.style.y,i.style._width=i.style.width,i.style._height=i.style.height,t)n(e,t,i,a,o);else{var s=i._x||0,l=i._y||0;i.scale=[.01,.01,s,l],e.addShape(i),e.animate(i.id,"").delay(r).when(a,{scale:[1,1,s,l]}).start(o||"QuinticOut")}}function c(e,t,n,a,o){t||(t={style:{xStart:n.style.xStart,yStart:n.style.yStart,xEnd:n.style.xStart,yEnd:n.style.yStart}});var r=n.style.xStart,s=n.style.xEnd,l=n.style.yStart,h=n.style.yEnd;i(n,t,"xStart","xEnd","yStart","yEnd"),e.addShape(n),e.animate(n.id,"style").when(a,{xStart:r,xEnd:s,yStart:l,yEnd:h}).start(o)}function p(e,t,i,n,a){i.style.smooth?t?e.addShape(i):(i.style.pointListLength=1,e.addShape(i),i.style.pointList=i.style.pointList||i.getPointList(i.style),e.animate(i.id,"style").when(n,{pointListLength:i.style.pointList.length}).start(a||"QuinticOut")):(i.style.pointList=t?t.style.pointList:[[i.style.xStart,i.style.yStart],[i.style.xStart,i.style.yStart]],e.addShape(i),e.animate(i.id,"style").when(n,{pointList:[[i.style.xStart,i.style.yStart],[i._x||0,i._y||0]]}).start(a||"QuinticOut"))}var u=e("zrender/tool/util");return{pointList:t,rectangle:n,candle:a,ring:o,sector:r,text:s,polygon:l,ribbon:h,gaugePointer:d,icon:m,line:c,markline:p}}),define("echarts/util/ecEffect",["require","../util/ecData","zrender/shape/Circle","zrender/shape/Image","../util/shape/Icon","../util/shape/Symbol","zrender/tool/env"],function(e){function t(e,t,i,n){var o=i.effect,l=o.color||i.style.strokeColor||i.style.color,d=o.shadowColor||l,m=o.scaleSize,c="undefined"!=typeof o.shadowBlur?o.shadowBlur:m,p=new s({zlevel:n,style:{brushType:"stroke",iconType:"pin"!=i.style.iconType&&"droplet"!=i.style.iconType?i.style.iconType:"circle",x:c+1,y:c+1,n:i.style.n,width:i.style._width*m,height:i.style._height*m,lineWidth:1,strokeColor:l,shadowColor:d,shadowBlur:c},draggable:!1,hoverable:!1});h&&(p.style.image=e.shapeToImage(p,p.style.width+2*c+2,p.style.height+2*c+2).style.image,p=new r({zlevel:p.zlevel,style:p.style,draggable:!1,hoverable:!1})),a.clone(i,p),p.position=i.position,t.push(p),e.addShape(p);var u=window.devicePixelRatio||1,V=(p.style.width/u-i.style._width)/2;p.style.x=i.style._x-V,p.style.y=i.style._y-V;var U=100*(o.period+10*Math.random());e.modShape(i.id,{invisible:!0});var y=p.style.x+p.style.width/2/u,g=p.style.y+p.style.height/2/u;e.modShape(p.id,{scale:[.1,.1,y,g]}),e.animate(p.id,"",o.loop).when(U,{scale:[1,1,y,g]}).done(function(){i.effect.show=!1,e.delShape(p.id)}).start()}function i(e,t,i,n){var a=i.effect,o=a.color||i.style.strokeColor||i.style.color,r=a.scaleSize,s=a.shadowColor||o,h="undefined"!=typeof a.shadowBlur?a.shadowBlur:2*r,d=window.devicePixelRatio||1,m=new l({zlevel:n,position:i.position,scale:i.scale,style:{pointList:i.style.pointList,iconType:i.style.iconType,color:o,strokeColor:o,shadowColor:s,shadowBlur:h*d,random:!0,brushType:"fill",lineWidth:1,size:i.style.size},draggable:!1,hoverable:!1});t.push(m),e.addShape(m),e.modShape(i.id,{invisible:!0});for(var c=Math.round(100*a.period),p={},u={},V=0;20>V;V++)m.style["randomMap"+V]=0,p={},p["randomMap"+V]=100,u={},u["randomMap"+V]=0,m.style["randomMap"+V]=100*Math.random(),e.animate(m.id,"style",!0).when(c,p).when(2*c,u).when(3*c,p).when(4*c,p).delay(Math.random()*c*V).start()}function n(e,t,i,n){var s,l=i.effect,d=l.color||i.style.strokeColor||i.style.color,m=l.shadowColor||i.style.strokeColor||d,c=i.style.lineWidth*l.scaleSize,p="undefined"!=typeof l.shadowBlur?l.shadowBlur:c,u=new o({zlevel:n,style:{x:p,y:p,r:c,color:d,shadowColor:m,shadowBlur:p},draggable:!1,hoverable:!1});h?(u.style.image=e.shapeToImage(u,2*(c+p),2*(c+p)).style.image,u=new r({zlevel:u.zlevel,style:u.style,draggable:!1,hoverable:!1}),s=p):s=0,a.clone(i,u),u.position=i.position,t.push(u),e.addShape(u),u.style.x=i.style.xStart-s,u.style.y=i.style.yStart-s;var V=(i.style.xStart-i.style.xEnd)*(i.style.xStart-i.style.xEnd)+(i.style.yStart-i.style.yEnd)*(i.style.yStart-i.style.yEnd),U=Math.round(Math.sqrt(Math.round(V*l.period*l.period)));if(i.style.smooth){var y=i.style.pointList||i.getPointList(i.style),g=y.length;U=Math.round(U/g);for(var f=e.animate(u.id,"style",l.loop),b=Math.ceil(g/8),_=0;g-b>_;_+=b)f.when(U*(_+1),{x:y[_][0]-s,y:y[_][1]-s});f.when(U*g,{x:y[g-1][0]-s,y:y[g-1][1]-s}),f.done(function(){i.effect.show=!1,e.delShape(u.id)}),f.start("spline")}else e.animate(u.id,"style",l.loop).when(U,{x:i._x-s,y:i._y-s}).done(function(){i.effect.show=!1,e.delShape(u.id)}).start()}var a=e("../util/ecData"),o=e("zrender/shape/Circle"),r=e("zrender/shape/Image"),s=e("../util/shape/Icon"),l=e("../util/shape/Symbol"),h=e("zrender/tool/env").canvasSupported;return{point:t,largePoint:i,line:n}}),define("zrender/shape/Star",["require","../tool/math","./Base","../tool/util"],function(e){var t=e("../tool/math"),i=t.sin,n=t.cos,a=Math.PI,o=e("./Base"),r=function(e){o.call(this,e)};return r.prototype={type:"star",buildPath:function(e,t){var o=t.n;if(o&&!(2>o)){var r=t.x,s=t.y,l=t.r,h=t.r0;null==h&&(h=o>4?l*n(2*a/o)/n(a/o):l/3);var d=a/o,m=-a/2,c=r+l*n(m),p=s+l*i(m);m+=d;var u=t.pointList=[];u.push([c,p]);for(var V,U=0,y=2*o-1;y>U;U++)V=U%2===0?h:l,u.push([r+V*n(m),s+V*i(m)]),m+=d;u.push([c,p]),e.moveTo(u[0][0],u[0][1]);for(var U=0;U=a.x&&e<=a.x+a.width&&t>=a.y&&t<=a.y+a.height?n.isInsidePath(this._pathProxy.pathCommands,this.style.lineWidth,this.style.brushType,e,t):void 0}},e("../tool/util").inherits(a,t),a}),define("zrender/shape/Droplet",["require","./Base","./util/PathProxy","zrender/tool/area","../tool/util"],function(e){"use strict";var t=e("./Base"),i=e("./util/PathProxy"),n=e("zrender/tool/area"),a=function(e){t.call(this,e),this._pathProxy=new i};return a.prototype={type:"droplet",buildPath:function(e,t){var n=this._pathProxy||new i;n.begin(e),n.moveTo(t.x,t.y+t.a),n.bezierCurveTo(t.x+t.a,t.y+t.a,t.x+3*t.a/2,t.y-t.a/3,t.x,t.y-t.b),n.bezierCurveTo(t.x-3*t.a/2,t.y-t.a/3,t.x-t.a,t.y+t.a,t.x,t.y+t.a),n.closePath()},getRect:function(e){return e.__rect?e.__rect:(this._pathProxy.isEmpty()||this.buildPath(null,e),this._pathProxy.fastBoundingRect())},isCover:function(e,t){var i=this.getTansform(e,t);e=i[0],t=i[1];var a=this.getRect(this.style);return e>=a.x&&e<=a.x+a.width&&t>=a.y&&t<=a.y+a.height?n.isInsidePath(this._pathProxy.pathCommands,this.style.lineWidth,this.style.brushType,e,t):void 0}},e("../tool/util").inherits(a,t),a}),define("zrender/tool/math",[],function(){function e(e,t){return Math.sin(t?e*a:e)}function t(e,t){return Math.cos(t?e*a:e)}function i(e){return e*a}function n(e){return e/a}var a=Math.PI/180;return{sin:e,cos:t,degreeToRadian:i,radianToDegree:n}}),define("zrender/shape/util/PathProxy",["require","../../tool/vector"],function(e){var t=e("../../tool/vector"),i=function(e,t){this.command=e,this.points=t||null},n=function(){this.pathCommands=[],this._ctx=null,this._min=[],this._max=[]};return n.prototype.fastBoundingRect=function(){var e=this._min,i=this._max;e[0]=e[1]=1/0,i[0]=i[1]=-1/0;for(var n=0;nr;r+=2)e[0]=Math.min(e[0],e[0],o[r]),e[1]=Math.min(e[1],e[1],o[r+1]),i[0]=Math.max(i[0],i[0],o[r]),i[1]=Math.max(i[1],i[1],o[r+1]);break;case"Q":for(var r=0;4>r;r+=2)e[0]=Math.min(e[0],e[0],o[r]),e[1]=Math.min(e[1],e[1],o[r+1]),i[0]=Math.max(i[0],i[0],o[r]),i[1]=Math.max(i[1],i[1],o[r+1]);break;case"A":var s=o[0],l=o[1],h=o[2],d=o[3];e[0]=Math.min(e[0],e[0],s-h),e[1]=Math.min(e[1],e[1],l-d),i[0]=Math.max(i[0],i[0],s+h),i[1]=Math.max(i[1],i[1],l+d)}}return{x:e[0],y:e[1],width:i[0]-e[0],height:i[1]-e[1]}},n.prototype.begin=function(e){return this._ctx=e||null,this.pathCommands.length=0,this},n.prototype.moveTo=function(e,t){return this.pathCommands.push(new i("M",[e,t])),this._ctx&&this._ctx.moveTo(e,t),this},n.prototype.lineTo=function(e,t){return this.pathCommands.push(new i("L",[e,t])),this._ctx&&this._ctx.lineTo(e,t),this},n.prototype.bezierCurveTo=function(e,t,n,a,o,r){return this.pathCommands.push(new i("C",[e,t,n,a,o,r])),this._ctx&&this._ctx.bezierCurveTo(e,t,n,a,o,r),this},n.prototype.quadraticCurveTo=function(e,t,n,a){return this.pathCommands.push(new i("Q",[e,t,n,a])),this._ctx&&this._ctx.quadraticCurveTo(e,t,n,a),this},n.prototype.arc=function(e,t,n,a,o,r){return this.pathCommands.push(new i("A",[e,t,n,n,a,o-a,0,r?0:1])),this._ctx&&this._ctx.arc(e,t,n,a,o,r),this},n.prototype.arcTo=function(e,t,i,n,a){return this._ctx&&this._ctx.arcTo(e,t,i,n,a),this},n.prototype.rect=function(e,t,i,n){return this._ctx&&this._ctx.rect(e,t,i,n),this},n.prototype.closePath=function(){return this.pathCommands.push(new i("z")),this._ctx&&this._ctx.closePath(),this},n.prototype.isEmpty=function(){return 0===this.pathCommands.length},n.PathSegment=i,n}),define("zrender/shape/Line",["require","./Base","./util/dashedLineTo","../tool/util"],function(e){var t=e("./Base"),i=e("./util/dashedLineTo"),n=function(e){this.brushTypeOnly="stroke",this.textPosition="end",t.call(this,e)};return n.prototype={type:"line",buildPath:function(e,t){if(t.lineType&&"solid"!=t.lineType){if("dashed"==t.lineType||"dotted"==t.lineType){var n=(t.lineWidth||1)*("dashed"==t.lineType?5:1);i(e,t.xStart,t.yStart,t.xEnd,t.yEnd,n)}}else e.moveTo(t.xStart,t.yStart),e.lineTo(t.xEnd,t.yEnd)},getRect:function(e){if(e.__rect)return e.__rect;var t=e.lineWidth||1;return e.__rect={x:Math.min(e.xStart,e.xEnd)-t,y:Math.min(e.yStart,e.yEnd)-t,width:Math.abs(e.xStart-e.xEnd)+t,height:Math.abs(e.yStart-e.yEnd)+t},e.__rect}},e("../tool/util").inherits(n,t),n}),define("zrender/shape/BrokenLine",["require","./Base","./util/smoothSpline","./util/smoothBezier","./util/dashedLineTo","./Polygon","../tool/util"],function(e){var t=e("./Base"),i=e("./util/smoothSpline"),n=e("./util/smoothBezier"),a=e("./util/dashedLineTo"),o=function(e){this.brushTypeOnly="stroke",this.textPosition="end",t.call(this,e)};return o.prototype={type:"broken-line",buildPath:function(e,t){var o=t.pointList;if(!(o.length<2)){var r=Math.min(t.pointList.length,Math.round(t.pointListLength||t.pointList.length));if(t.smooth&&"spline"!==t.smooth){var s=n(o,t.smooth,!1,t.smoothConstraint);e.moveTo(o[0][0],o[0][1]);for(var l,h,d,m=0;r-1>m;m++)l=s[2*m],h=s[2*m+1],d=o[m+1],e.bezierCurveTo(l[0],l[1],h[0],h[1],d[0],d[1])}else if("spline"===t.smooth&&(o=i(o),r=o.length),t.lineType&&"solid"!=t.lineType){if("dashed"==t.lineType||"dotted"==t.lineType){var c=(t.lineWidth||1)*("dashed"==t.lineType?5:1);e.moveTo(o[0][0],o[0][1]);for(var m=1;r>m;m++)a(e,o[m-1][0],o[m-1][1],o[m][0],o[m][1],c)}}else{e.moveTo(o[0][0],o[0][1]);for(var m=1;r>m;m++)e.lineTo(o[m][0],o[m][1])}}},getRect:function(t){return e("./Polygon").prototype.getRect(t)}},e("../tool/util").inherits(o,t),o}),define("zrender/shape/util/dashedLineTo",[],function(){var e=[5,5];return function(t,i,n,a,o,r){if(t.setLineDash)return e[0]=e[1]=r,t.setLineDash(e),t.moveTo(i,n),void t.lineTo(a,o);r="number"!=typeof r?5:r;var s=a-i,l=o-n,h=Math.floor(Math.sqrt(s*s+l*l)/r);s/=h,l/=h;for(var d=!0,m=0;h>m;++m)d?t.moveTo(i,n):t.lineTo(i,n),d=!d,i+=s,n+=l;t.lineTo(a,o)}}),define("zrender/shape/util/smoothSpline",["require","../../tool/vector"],function(e){function t(e,t,i,n,a,o,r){var s=.5*(i-e),l=.5*(n-t);return(2*(t-i)+s+l)*r+(-3*(t-i)-2*s-l)*o+s*a+t}var i=e("../../tool/vector");return function(e,n){for(var a=e.length,o=[],r=0,s=1;a>s;s++)r+=i.distance(e[s-1],e[s]);var l=r/5;l=a>l?a:l;for(var s=0;l>s;s++){var h,d,m,c=s/(l-1)*(n?a:a-1),p=Math.floor(c),u=c-p,V=e[p%a];n?(h=e[(p-1+a)%a],d=e[(p+1)%a],m=e[(p+2)%a]):(h=e[0===p?p:p-1],d=e[p>a-2?a-1:p+1],m=e[p>a-3?a-1:p+2]);var U=u*u,y=u*U;o.push([t(h[0],V[0],d[0],m[0],u,U,y),t(h[1],V[1],d[1],m[1],u,U,y)])}return o}}),define("zrender/shape/util/smoothBezier",["require","../../tool/vector"],function(e){var t=e("../../tool/vector");return function(e,i,n,a){var o,r,s,l,h=[],d=[],m=[],c=[],p=!!a;if(p){s=[1/0,1/0],l=[-1/0,-1/0];for(var u=0,V=e.length;V>u;u++)t.min(s,s,e[u]),t.max(l,l,e[u]);t.min(s,s,a[0]),t.max(l,l,a[1])}for(var u=0,V=e.length;V>u;u++){var o,r,U=e[u];if(n)o=e[u?u-1:V-1],r=e[(u+1)%V];else{if(0===u||u===V-1){h.push(e[u]);continue}o=e[u-1],r=e[u+1]}t.sub(d,r,o),t.scale(d,d,i);var y=t.distance(U,o),g=t.distance(U,r),f=y+g;0!==f&&(y/=f,g/=f),t.scale(m,d,-y),t.scale(c,d,g);var b=t.add([],U,m),_=t.add([],U,c);p&&(t.max(b,b,s),t.min(b,b,l),t.max(_,_,s),t.min(_,_,l)),h.push(b),h.push(_)}return n&&h.push(h.shift()),h}}),define("zrender/shape/Polygon",["require","./Base","./util/smoothSpline","./util/smoothBezier","./util/dashedLineTo","../tool/util"],function(e){var t=e("./Base"),i=e("./util/smoothSpline"),n=e("./util/smoothBezier"),a=e("./util/dashedLineTo"),o=function(e){t.call(this,e)};return o.prototype={type:"polygon",brush:function(e,t){var i=this.style;t&&(i=this.getHighlightStyle(i,this.highlightStyle||{})),e.save(),this.setContext(e,i),this.setTransform(e);var n=!1;("fill"==i.brushType||"both"==i.brushType||"undefined"==typeof i.brushType)&&(e.beginPath(),"dashed"==i.lineType||"dotted"==i.lineType?(this.buildPath(e,{lineType:"solid",lineWidth:i.lineWidth,pointList:i.pointList}),n=!1):(this.buildPath(e,i),n=!0),e.closePath(),e.fill()),i.lineWidth>0&&("stroke"==i.brushType||"both"==i.brushType)&&(n||(e.beginPath(),this.buildPath(e,i)),e.stroke()),this.drawText(e,i,this.style),e.restore()},buildPath:function(e,t){var o=t.pointList;if(!(o.length<2))if(t.smooth&&"spline"!==t.smooth){var r=n(o,t.smooth,!0,t.smoothConstraint);e.moveTo(o[0][0],o[0][1]);for(var s,l,h,d=o.length,m=0;d>m;m++)s=r[2*m],l=r[2*m+1],h=o[(m+1)%d],e.bezierCurveTo(s[0],s[1],l[0],l[1],h[0],h[1])}else if("spline"===t.smooth&&(o=i(o,!0)),t.lineType&&"solid"!=t.lineType){if("dashed"==t.lineType||"dotted"==t.lineType){var c=t._dashLength||(t.lineWidth||1)*("dashed"==t.lineType?5:1);t._dashLength=c,e.moveTo(o[0][0],o[0][1]);for(var m=1,p=o.length;p>m;m++)a(e,o[m-1][0],o[m-1][1],o[m][0],o[m][1],c);a(e,o[o.length-1][0],o[o.length-1][1],o[0][0],o[0][1],c)}}else{e.moveTo(o[0][0],o[0][1]);for(var m=1,p=o.length;p>m;m++)e.lineTo(o[m][0],o[m][1]);e.lineTo(o[0][0],o[0][1])}},getRect:function(e){if(e.__rect)return e.__rect;for(var t=Number.MAX_VALUE,i=Number.MIN_VALUE,n=Number.MAX_VALUE,a=Number.MIN_VALUE,o=e.pointList,r=0,s=o.length;s>r;r++)o[r][0]i&&(i=o[r][0]),o[r][1]a&&(a=o[r][1]);var l;return l="stroke"==e.brushType||"fill"==e.brushType?e.lineWidth||1:0,e.__rect={x:Math.round(t-l/2),y:Math.round(n-l/2),width:i-t+l,height:a-n+l},e.__rect}},e("../tool/util").inherits(o,t),o}),define("echarts/util/shape/normalIsCover",[],function(){return function(e,t){var i=this.getTansform(e,t);e=i[0],t=i[1];var n=this.style.__rect;return n||(n=this.style.__rect=this.getRect(this.style)),e>=n.x&&e<=n.x+n.width&&t>=n.y&&t<=n.y+n.height}}),define("echarts/component/dataView",["require","./base","../config","zrender/tool/util","../component"],function(e){function t(e,t,n,a,o){i.call(this,e,t,n,a,o),this.dom=o.dom,this._tDom=document.createElement("div"),this._textArea=document.createElement("textArea"),this._buttonRefresh=document.createElement("button"),this._buttonClose=document.createElement("button"),this._hasShow=!1,this._zrHeight=n.getHeight(),this._zrWidth=n.getWidth(),this._tDom.className="echarts-dataview",this.hide(),this.dom.firstChild.appendChild(this._tDom),window.addEventListener?(this._tDom.addEventListener("click",this._stop),this._tDom.addEventListener("mousewheel",this._stop),this._tDom.addEventListener("mousemove",this._stop),this._tDom.addEventListener("mousedown",this._stop),this._tDom.addEventListener("mouseup",this._stop),this._tDom.addEventListener("touchstart",this._stop),this._tDom.addEventListener("touchmove",this._stop),this._tDom.addEventListener("touchend",this._stop)):(this._tDom.attachEvent("onclick",this._stop),this._tDom.attachEvent("onmousewheel",this._stop),this._tDom.attachEvent("onmousemove",this._stop),this._tDom.attachEvent("onmousedown",this._stop),this._tDom.attachEvent("onmouseup",this._stop))}var i=e("./base"),n=e("../config"),a=e("zrender/tool/util");return t.prototype={type:n.COMPONENT_TYPE_DATAVIEW,_lang:["Data View","close","refresh"],_gCssText:"position:absolute;display:block;overflow:hidden;transition:height 0.8s,background-color 1s;-moz-transition:height 0.8s,background-color 1s;-webkit-transition:height 0.8s,background-color 1s;-o-transition:height 0.8s,background-color 1s;z-index:1;left:0;top:0;",hide:function(){this._sizeCssText="width:"+this._zrWidth+"px;height:0px;background-color:#f0ffff;",this._tDom.style.cssText=this._gCssText+this._sizeCssText},show:function(e){this._hasShow=!0;var t=this.query(this.option,"toolbox.feature.dataView.lang")||this._lang;this.option=e,this._tDom.innerHTML='

'+(t[0]||this._lang[0])+"

",this._textArea.style.cssText="display:block;margin:0 0 8px 0;padding:4px 6px;overflow:auto;width:"+(this._zrWidth-15)+"px;height:"+(this._zrHeight-100)+"px;";var i=this.query(this.option,"toolbox.feature.dataView.optionToContent");this._textArea.value="function"!=typeof i?this._optionToContent():i(this.option),this._tDom.appendChild(this._textArea),this._buttonClose.style.cssText="float:right;padding:1px 6px;",this._buttonClose.innerHTML=t[1]||this._lang[1];var n=this;this._buttonClose.onclick=function(){n.hide()},this._tDom.appendChild(this._buttonClose),this.query(this.option,"toolbox.feature.dataView.readOnly")===!1?(this._buttonRefresh.style.cssText="float:right;margin-right:10px;padding:1px 6px;",this._buttonRefresh.innerHTML=t[2]||this._lang[2],this._buttonRefresh.onclick=function(){n._save()},this._tDom.appendChild(this._buttonRefresh),this._textArea.readOnly=!1,this._textArea.style.cursor="default"):(this._textArea.readOnly=!0,this._textArea.style.cursor="text"),this._sizeCssText="width:"+this._zrWidth+"px;height:"+this._zrHeight+"px;background-color:#fff;",this._tDom.style.cssText=this._gCssText+this._sizeCssText},_optionToContent:function(){var e,t,i,a,o,r,s=[],l="";if(this.option.xAxis)for(s=this.option.xAxis instanceof Array?this.option.xAxis:[this.option.xAxis],e=0,a=s.length;a>e;e++)if("category"==(s[e].type||"category")){for(r=[],t=0,i=s[e].data.length;i>t;t++)o=s[e].data[t],r.push("undefined"!=typeof o.value?o.value:o);l+=r.join(", ")+"\n\n"}if(this.option.yAxis)for(s=this.option.yAxis instanceof Array?this.option.yAxis:[this.option.yAxis],e=0,a=s.length;a>e;e++)if("category"==s[e].type){for(r=[],t=0,i=s[e].data.length;i>t;t++)o=s[e].data[t],r.push("undefined"!=typeof o.value?o.value:o);l+=r.join(", ")+"\n\n"}var h,d=this.option.series;for(e=0,a=d.length;a>e;e++){for(r=[],t=0,i=d[e].data.length;i>t;t++)o=d[e].data[t],h=d[e].type==n.CHART_TYPE_PIE||d[e].type==n.CHART_TYPE_MAP?(o.name||"-")+":":"",d[e].type==n.CHART_TYPE_SCATTER&&(o="undefined"!=typeof o.value?o.value:o,o=o.join(", ")),r.push(h+("undefined"!=typeof o.value?o.value:o));l+=(d[e].name||"-")+" : \n",l+=r.join(d[e].type==n.CHART_TYPE_SCATTER?"\n":", "),l+="\n\n"}return l},_save:function(){var e=this._textArea.value,t=this.query(this.option,"toolbox.feature.dataView.contentToOption");if("function"!=typeof t){e=e.split("\n");for(var i=[],a=0,o=e.length;o>a;a++)e[a]=this._trim(e[a]),""!==e[a]&&i.push(e[a]);this._contentToOption(i)}else t(e,this.option);this.hide();var r=this;setTimeout(function(){r.messageCenter&&r.messageCenter.dispatch(n.EVENT.DATA_VIEW_CHANGED,null,{option:r.option},r.myChart)},r.canvasSupported?800:100)},_contentToOption:function(e){var t,i,a,o,r,s,l,h=[],d=0;if(this.option.xAxis)for(h=this.option.xAxis instanceof Array?this.option.xAxis:[this.option.xAxis],t=0,o=h.length;o>t;t++)if("category"==(h[t].type||"category")){for(s=e[d].split(","),i=0,a=h[t].data.length;a>i;i++)l=this._trim(s[i]||""),r=h[t].data[i],"undefined"!=typeof h[t].data[i].value?h[t].data[i].value=l:h[t].data[i]=l;d++}if(this.option.yAxis)for(h=this.option.yAxis instanceof Array?this.option.yAxis:[this.option.yAxis],t=0,o=h.length;o>t;t++)if("category"==h[t].type){for(s=e[d].split(","),i=0,a=h[t].data.length;a>i;i++)l=this._trim(s[i]||""),r=h[t].data[i],"undefined"!=typeof h[t].data[i].value?h[t].data[i].value=l:h[t].data[i]=l;d++}var m=this.option.series;for(t=0,o=m.length;o>t;t++)if(d++,m[t].type==n.CHART_TYPE_SCATTER)for(var i=0,a=m[t].data.length;a>i;i++)s=e[d],l=s.replace(" ","").split(","),"undefined"!=typeof m[t].data[i].value?m[t].data[i].value=l:m[t].data[i]=l,d++;else{s=e[d].split(",");for(var i=0,a=m[t].data.length;a>i;i++)l=(s[i]||"").replace(/.*:/,""),l=this._trim(l),l="-"!=l&&""!==l?l-0:"-","undefined"!=typeof m[t].data[i].value?m[t].data[i].value=l:m[t].data[i]=l;d++}},_trim:function(e){var t=new RegExp("(^[\\s\\t\\xa0\\u3000]+)|([\\u3000\\xa0\\s\\t]+$)","g");return e.replace(t,"")},_stop:function(e){e=e||window.event,e.stopPropagation?e.stopPropagation():e.cancelBubble=!0},resize:function(){this._zrHeight=this.zr.getHeight(),this._zrWidth=this.zr.getWidth(),this._tDom.offsetHeight>10&&(this._sizeCssText="width:"+this._zrWidth+"px;height:"+this._zrHeight+"px;background-color:#fff;",this._tDom.style.cssText=this._gCssText+this._sizeCssText,this._textArea.style.cssText="display:block;margin:0 0 8px 0;padding:4px 6px;overflow:auto;width:"+(this._zrWidth-15)+"px;height:"+(this._zrHeight-100)+"px;")},dispose:function(){window.removeEventListener?(this._tDom.removeEventListener("click",this._stop),this._tDom.removeEventListener("mousewheel",this._stop),this._tDom.removeEventListener("mousemove",this._stop),this._tDom.removeEventListener("mousedown",this._stop),this._tDom.removeEventListener("mouseup",this._stop),this._tDom.removeEventListener("touchstart",this._stop),this._tDom.removeEventListener("touchmove",this._stop),this._tDom.removeEventListener("touchend",this._stop)):(this._tDom.detachEvent("onclick",this._stop),this._tDom.detachEvent("onmousewheel",this._stop),this._tDom.detachEvent("onmousemove",this._stop),this._tDom.detachEvent("onmousedown",this._stop),this._tDom.detachEvent("onmouseup",this._stop)),this._buttonRefresh.onclick=null,this._buttonClose.onclick=null,this._hasShow&&(this._tDom.removeChild(this._textArea),this._tDom.removeChild(this._buttonRefresh),this._tDom.removeChild(this._buttonClose)),this._textArea=null,this._buttonRefresh=null,this._buttonClose=null,this.dom.firstChild.removeChild(this._tDom),this._tDom=null}},a.inherits(t,i),e("../component").define("dataView",t),t}),define("echarts/util/shape/Cross",["require","zrender/shape/Base","zrender/shape/Line","zrender/tool/util","./normalIsCover"],function(e){function t(e){i.call(this,e)}var i=e("zrender/shape/Base"),n=e("zrender/shape/Line"),a=e("zrender/tool/util");return t.prototype={type:"cross",buildPath:function(e,t){var i=t.rect;t.xStart=i.x,t.xEnd=i.x+i.width,t.yStart=t.yEnd=t.y,n.prototype.buildPath(e,t),t.xStart=t.xEnd=t.x,t.yStart=i.y,t.yEnd=i.y+i.height,n.prototype.buildPath(e,t)},getRect:function(e){return e.rect},isCover:e("./normalIsCover")},a.inherits(t,i),t}),define("zrender/shape/Sector",["require","../tool/math","../tool/computeBoundingBox","../tool/vector","./Base","../tool/util"],function(e){var t=e("../tool/math"),i=e("../tool/computeBoundingBox"),n=e("../tool/vector"),a=e("./Base"),o=n.create(),r=n.create(),s=n.create(),l=n.create(),h=function(e){a.call(this,e)};return h.prototype={type:"sector",buildPath:function(e,i){var n=i.x,a=i.y,o=i.r0||0,r=i.r,s=i.startAngle,l=i.endAngle,h=i.clockWise||!1;s=t.degreeToRadian(s),l=t.degreeToRadian(l),h||(s=-s,l=-l);var d=t.cos(s),m=t.sin(s);e.moveTo(d*o+n,m*o+a),e.lineTo(d*r+n,m*r+a),e.arc(n,a,r,s,l,!h),e.lineTo(t.cos(l)*o+n,t.sin(l)*o+a),0!==o&&e.arc(n,a,o,l,s,h),e.closePath()},getRect:function(e){if(e.__rect)return e.__rect;var a=e.x,h=e.y,d=e.r0||0,m=e.r,c=t.degreeToRadian(e.startAngle),p=t.degreeToRadian(e.endAngle),u=e.clockWise;return u||(c=-c,p=-p),d>1?i.arc(a,h,d,c,p,!u,o,s):(o[0]=s[0]=a,o[1]=s[1]=h),i.arc(a,h,m,c,p,!u,r,l),n.min(o,o,r),n.max(s,s,l),e.__rect={x:o[0],y:o[1],width:s[0]-o[0],height:s[1]-o[1]},e.__rect}},e("../tool/util").inherits(h,a),h}),define("echarts/util/shape/Candle",["require","zrender/shape/Base","zrender/tool/util","./normalIsCover"],function(e){function t(e){i.call(this,e)}var i=e("zrender/shape/Base"),n=e("zrender/tool/util");return t.prototype={type:"candle",_numberOrder:function(e,t){return t-e},buildPath:function(e,t){var i=n.clone(t.y).sort(this._numberOrder);e.moveTo(t.x,i[3]),e.lineTo(t.x,i[2]),e.moveTo(t.x-t.width/2,i[2]),e.rect(t.x-t.width/2,i[2],t.width,i[1]-i[2]),e.moveTo(t.x,i[1]),e.lineTo(t.x,i[0])},getRect:function(e){if(!e.__rect){var t=0;("stroke"==e.brushType||"fill"==e.brushType)&&(t=e.lineWidth||1);var i=n.clone(e.y).sort(this._numberOrder);e.__rect={x:Math.round(e.x-e.width/2-t/2),y:Math.round(i[3]-t/2),width:e.width+t,height:i[0]-i[3]+t}}return e.__rect},isCover:e("./normalIsCover")},n.inherits(t,i),t}),define("zrender/tool/computeBoundingBox",["require","./vector","./curve"],function(e){function t(e,t,i){if(0!==e.length){for(var n=e[0][0],a=e[0][0],o=e[0][1],r=e[0][1],s=1;sa&&(a=l[0]),l[1]r&&(r=l[1])}t[0]=n,t[1]=o,i[0]=a,i[1]=r}}function i(e,t,i,n,a,r){var s=[];o.cubicExtrema(e[0],t[0],i[0],n[0],s);for(var l=0;ln&&(n+=2*Math.PI),o%=2*Math.PI,0>o&&(o+=2*Math.PI),n>o&&!h?o+=2*Math.PI:o>n&&h&&(n+=2*Math.PI),h){var c=o;o=n,n=c}for(var p=0;o>p;p+=Math.PI/2)p>n&&(l[0]=Math.cos(p)*i+e,l[1]=Math.sin(p)*i+t,a.min(d,l,d),a.max(m,l,m))};return t.cubeBezier=i,t.quadraticBezier=n,t.arc=h,t}),define("echarts/util/shape/Chain",["require","zrender/shape/Base","./Icon","zrender/shape/util/dashedLineTo","zrender/tool/util","zrender/tool/matrix"],function(e){function t(e){i.call(this,e)}var i=e("zrender/shape/Base"),n=e("./Icon"),a=e("zrender/shape/util/dashedLineTo"),o=e("zrender/tool/util"),r=e("zrender/tool/matrix");return t.prototype={type:"chain",brush:function(e,t){var i=this.style;t&&(i=this.getHighlightStyle(i,this.highlightStyle||{})),e.save(),this.setContext(e,i),this.setTransform(e),e.save(),e.beginPath(),this.buildLinePath(e,i),e.stroke(),e.restore(),this.brushSymbol(e,i),e.restore()},buildLinePath:function(e,t){var i=t.x,n=t.y+5,o=t.width,r=t.height/2-10;if(e.moveTo(i,n),e.lineTo(i,n+r),e.moveTo(i+o,n),e.lineTo(i+o,n+r),e.moveTo(i,n+r/2),t.lineType&&"solid"!=t.lineType){if("dashed"==t.lineType||"dotted"==t.lineType){var s=(t.lineWidth||1)*("dashed"==t.lineType?5:1);a(e,i,n+r/2,i+o,n+r/2,s)}}else e.lineTo(i+o,n+r/2)},brushSymbol:function(e,t){var i=t.y+t.height/4;e.save();for(var a,o=t.chainPoint,r=0,s=o.length;s>r;r++){if(a=o[r],"none"!=a.symbol){e.beginPath();var l=a.symbolSize;n.prototype.buildPath(e,{iconType:a.symbol,x:a.x-l,y:i-l,width:2*l,height:2*l,n:a.n}),e.fillStyle=a.isEmpty?"#fff":t.strokeColor,e.closePath(),e.fill(),e.stroke()}a.showLabel&&(e.font=a.textFont,e.fillStyle=a.textColor,e.textAlign=a.textAlign,e.textBaseline=a.textBaseline,a.rotation?(e.save(),this._updateTextTransform(e,a.rotation),e.fillText(a.name,a.textX,a.textY),e.restore()):e.fillText(a.name,a.textX,a.textY))}e.restore()},_updateTextTransform:function(e,t){var i=r.create();if(r.identity(i),0!==t[0]){var n=t[1]||0,a=t[2]||0;(n||a)&&r.translate(i,i,[-n,-a]),r.rotate(i,i,t[0]),(n||a)&&r.translate(i,i,[n,a])}e.transform.apply(e,i)},isCover:function(e,t){var i=this.style;return e>=i.x&&e<=i.x+i.width&&t>=i.y&&t<=i.y+i.height?!0:!1}},o.inherits(t,i),t}),define("zrender/shape/Ring",["require","./Base","../tool/util"],function(e){var t=e("./Base"),i=function(e){t.call(this,e)};return i.prototype={type:"ring",buildPath:function(e,t){e.arc(t.x,t.y,t.r,0,2*Math.PI,!1),e.moveTo(t.x+t.r0,t.y),e.arc(t.x,t.y,t.r0,0,2*Math.PI,!0)},getRect:function(e){if(e.__rect)return e.__rect;var t;return t="stroke"==e.brushType||"fill"==e.brushType?e.lineWidth||1:0,e.__rect={x:Math.round(e.x-e.r-t/2),y:Math.round(e.y-e.r-t/2),width:2*e.r+t,height:2*e.r+t},e.__rect}},e("../tool/util").inherits(i,t),i}),define("echarts/component/axis",["require","./base","zrender/shape/Line","../config","../util/ecData","zrender/tool/util","zrender/tool/color","./categoryAxis","./valueAxis","../component"],function(e){function t(e,t,n,a,o,r){i.call(this,e,t,n,a,o),this.axisType=r,this._axisList=[],this.refresh(a)}var i=e("./base"),n=e("zrender/shape/Line"),a=e("../config"),o=e("../util/ecData"),r=e("zrender/tool/util"),s=e("zrender/tool/color");return t.prototype={type:a.COMPONENT_TYPE_AXIS,axisBase:{_buildAxisLine:function(){var e=this.option.axisLine.lineStyle.width,t=e/2,i={_axisShape:"axisLine",zlevel:this._zlevelBase+1,hoverable:!1};switch(this.option.position){case"left":i.style={xStart:this.grid.getX()-t,yStart:this.grid.getYend(),xEnd:this.grid.getX()-t,yEnd:this.grid.getY(),lineCap:"round"};break;case"right":i.style={xStart:this.grid.getXend()+t,yStart:this.grid.getYend(),xEnd:this.grid.getXend()+t,yEnd:this.grid.getY(),lineCap:"round"};break;case"bottom":i.style={xStart:this.grid.getX(),yStart:this.grid.getYend()+t,xEnd:this.grid.getXend(),yEnd:this.grid.getYend()+t,lineCap:"round"};break;case"top":i.style={xStart:this.grid.getX(),yStart:this.grid.getY()-t,xEnd:this.grid.getXend(),yEnd:this.grid.getY()-t,lineCap:"round"}}""!==this.option.name&&(i.style.text=this.option.name,i.style.textPosition=this.option.nameLocation,i.style.textFont=this.getFont(this.option.nameTextStyle),this.option.nameTextStyle.align&&(i.style.textAlign=this.option.nameTextStyle.align),this.option.nameTextStyle.baseline&&(i.style.textBaseline=this.option.nameTextStyle.baseline),this.option.nameTextStyle.color&&(i.style.textColor=this.option.nameTextStyle.color)),i.style.strokeColor=this.option.axisLine.lineStyle.color,i.style.lineWidth=e,this.isHorizontal()?i.style.yStart=i.style.yEnd=this.subPixelOptimize(i.style.yEnd,e):i.style.xStart=i.style.xEnd=this.subPixelOptimize(i.style.xEnd,e),i.style.lineType=this.option.axisLine.lineStyle.type,i=new n(i),this.shapeList.push(i)},_axisLabelClickable:function(e,t){return e?(o.pack(t,void 0,-1,void 0,-1,t.style.text),t.hoverable=!0,t.clickable=!0,t.highlightStyle={color:s.lift(t.style.color,1),brushType:"fill"},t):t},refixAxisShape:function(e,t){if(this.option.axisLine.onZero){var i;if(this.isHorizontal()&&null!=t)for(var n=0,a=this.shapeList.length;a>n;n++)"axisLine"===this.shapeList[n]._axisShape?(this.shapeList[n].style.yStart=this.shapeList[n].style.yEnd=this.subPixelOptimize(t,this.shapeList[n].stylelineWidth),this.zr.modShape(this.shapeList[n].id)):"axisTick"===this.shapeList[n]._axisShape&&(i=this.shapeList[n].style.yEnd-this.shapeList[n].style.yStart,this.shapeList[n].style.yStart=t-i,this.shapeList[n].style.yEnd=t,this.zr.modShape(this.shapeList[n].id));if(!this.isHorizontal()&&null!=e)for(var n=0,a=this.shapeList.length;a>n;n++)"axisLine"===this.shapeList[n]._axisShape?(this.shapeList[n].style.xStart=this.shapeList[n].style.xEnd=this.subPixelOptimize(e,this.shapeList[n].stylelineWidth),this.zr.modShape(this.shapeList[n].id)):"axisTick"===this.shapeList[n]._axisShape&&(i=this.shapeList[n].style.xEnd-this.shapeList[n].style.xStart,this.shapeList[n].style.xStart=e,this.shapeList[n].style.xEnd=e+i,this.zr.modShape(this.shapeList[n].id))}},getPosition:function(){return this.option.position},isHorizontal:function(){return"bottom"===this.option.position||"top"===this.option.position}},reformOption:function(e){if(!e||e instanceof Array&&0===e.length?e=[{type:a.COMPONENT_TYPE_AXIS_VALUE}]:e instanceof Array||(e=[e]),e.length>2&&(e=[e[0],e[1]]),"xAxis"===this.axisType){(!e[0].position||"bottom"!=e[0].position&&"top"!=e[0].position)&&(e[0].position="bottom"),e.length>1&&(e[1].position="bottom"===e[0].position?"top":"bottom");for(var t=0,i=e.length;i>t;t++)e[t].type=e[t].type||"category",e[t].xAxisIndex=t,e[t].yAxisIndex=-1}else{(!e[0].position||"left"!=e[0].position&&"right"!=e[0].position)&&(e[0].position="left"),e.length>1&&(e[1].position="left"===e[0].position?"right":"left");for(var t=0,i=e.length;i>t;t++)e[t].type=e[t].type||"value",e[t].xAxisIndex=-1,e[t].yAxisIndex=t}return e},refresh:function(t){var i;t&&(this.option=t,"xAxis"===this.axisType?(this.option.xAxis=this.reformOption(t.xAxis),i=this.option.xAxis):(this.option.yAxis=this.reformOption(t.yAxis),i=this.option.yAxis),this.series=t.series);for(var n=e("./categoryAxis"),a=e("./valueAxis"),o=Math.max(i&&i.length||0,this._axisList.length),r=0;o>r;r++)!this._axisList[r]||!t||i[r]&&this._axisList[r].type==i[r].type||(this._axisList[r].dispose&&this._axisList[r].dispose(),this._axisList[r]=!1),this._axisList[r]?this._axisList[r].refresh&&this._axisList[r].refresh(i?i[r]:!1,this.series):i&&i[r]&&(this._axisList[r]="category"===i[r].type?new n(this.ecTheme,this.messageCenter,this.zr,i[r],this.myChart,this.axisBase):new a(this.ecTheme,this.messageCenter,this.zr,i[r],this.myChart,this.axisBase,this.series))},getAxis:function(e){return this._axisList[e]},clear:function(){for(var e=0,t=this._axisList.length;t>e;e++)this._axisList[e].dispose&&this._axisList[e].dispose();this._axisList=[]}},r.inherits(t,i),e("../component").define("axis",t),t}),define("echarts/component/grid",["require","./base","zrender/shape/Rectangle","../config","zrender/tool/util","../component"],function(e){function t(e,t,n,a,o){i.call(this,e,t,n,a,o),this.refresh(a)}var i=e("./base"),n=e("zrender/shape/Rectangle"),a=e("../config"),o=e("zrender/tool/util");return t.prototype={type:a.COMPONENT_TYPE_GRID,getX:function(){return this._x},getY:function(){return this._y},getWidth:function(){return this._width},getHeight:function(){return this._height},getXend:function(){return this._x+this._width},getYend:function(){return this._y+this._height},getArea:function(){return{x:this._x,y:this._y,width:this._width,height:this._height}},getBbox:function(){return[[this._x,this._y],[this.getXend(),this.getYend()]]},refixAxisShape:function(e){for(var t,i,n,o=e.xAxis._axisList.concat(e.yAxis?e.yAxis._axisList:[]),r=o.length;r--;)n=o[r],n.type==a.COMPONENT_TYPE_AXIS_VALUE&&n._min<0&&n._max>=0&&(n.isHorizontal()?t=n.getCoord(0):i=n.getCoord(0));if("undefined"!=typeof t||"undefined"!=typeof i)for(r=o.length;r--;)o[r].refixAxisShape(t,i)},refresh:function(e){if(e||this._zrWidth!=this.zr.getWidth()||this._zrHeight!=this.zr.getHeight()){this.clear(),this.option=e||this.option,this.option.grid=this.reformOption(this.option.grid);var t=this.option.grid;this._zrWidth=this.zr.getWidth(),this._zrHeight=this.zr.getHeight(),this._x=this.parsePercent(t.x,this._zrWidth),this._y=this.parsePercent(t.y,this._zrHeight);var i=this.parsePercent(t.x2,this._zrWidth),a=this.parsePercent(t.y2,this._zrHeight);this._width="undefined"==typeof t.width?this._zrWidth-this._x-i:this.parsePercent(t.width,this._zrWidth),this._width=this._width<=0?10:this._width,this._height="undefined"==typeof t.height?this._zrHeight-this._y-a:this.parsePercent(t.height,this._zrHeight),this._height=this._height<=0?10:this._height,this._x=this.subPixelOptimize(this._x,t.borderWidth),this._y=this.subPixelOptimize(this._y,t.borderWidth),this.shapeList.push(new n({zlevel:this._zlevelBase,hoverable:!1,style:{x:this._x,y:this._y,width:this._width,height:this._height,brushType:t.borderWidth>0?"both":"fill",color:t.backgroundColor,strokeColor:t.borderColor,lineWidth:t.borderWidth}})),this.zr.addShape(this.shapeList[0])}}},o.inherits(t,i),e("../component").define("grid",t),t}),define("echarts/component/dataZoom",["require","./base","zrender/shape/Rectangle","zrender/shape/Polygon","../util/shape/Icon","../config","../util/date","zrender/tool/util","../component"],function(e){function t(e,t,n,a,o){i.call(this,e,t,n,a,o);var r=this;r._ondrift=function(e,t){return r.__ondrift(this,e,t)},r._ondragend=function(){return r.__ondragend()},this._fillerSize=28,this._handleSize=8,this._isSilence=!1,this._zoom={},this.option.dataZoom=this.reformOption(this.option.dataZoom),this.zoomOption=this.option.dataZoom,this.myChart.canvasSupported||(this.zoomOption.realtime=!1),this._location=this._getLocation(),this._zoom=this._getZoom(),this._backupData(),this.option.dataZoom.show&&this._buildShape(),this._syncData()}var i=e("./base"),n=e("zrender/shape/Rectangle"),a=e("zrender/shape/Polygon"),o=e("../util/shape/Icon"),r=e("../config"),s=e("../util/date"),l=e("zrender/tool/util");return t.prototype={type:r.COMPONENT_TYPE_DATAZOOM,_buildShape:function(){this._buildBackground(),this._buildFiller(),this._buildHandle(),this._buildFrame();for(var e=0,t=this.shapeList.length;t>e;e++)this.zr.addShape(this.shapeList[e]);this._syncFrameShape()},_getLocation:function(){var e,t,i,n,a=this.component.grid;return"horizontal"==this.zoomOption.orient?(i=this.zoomOption.width||a.getWidth(),n=this.zoomOption.height||this._fillerSize,e=null!=this.zoomOption.x?this.zoomOption.x:a.getX(),t=null!=this.zoomOption.y?this.zoomOption.y:this.zr.getHeight()-n-2):(i=this.zoomOption.width||this._fillerSize,n=this.zoomOption.height||a.getHeight(),e=null!=this.zoomOption.x?this.zoomOption.x:2,t=null!=this.zoomOption.y?this.zoomOption.y:a.getY()),{x:e,y:t,width:i,height:n}},_getZoom:function(){var e=this.option.series,t=this.option.xAxis;!t||t instanceof Array||(t=[t],this.option.xAxis=t);var i=this.option.yAxis;!i||i instanceof Array||(i=[i],this.option.yAxis=i);var n,a,o=[],s=this.zoomOption.xAxisIndex;if(t&&null==s){n=[];for(var l=0,h=t.length;h>l;l++)("category"==t[l].type||null==t[l].type)&&n.push(l)}else n=s instanceof Array?s:null!=s?[s]:[];if(s=this.zoomOption.yAxisIndex,i&&null==s){a=[];for(var l=0,h=i.length;h>l;l++)"category"==i[l].type&&a.push(l)}else a=s instanceof Array?s:null!=s?[s]:[];for(var d,l=0,h=e.length;h>l;l++)if(d=e[l],d.type==r.CHART_TYPE_LINE||d.type==r.CHART_TYPE_BAR||d.type==r.CHART_TYPE_SCATTER||d.type==r.CHART_TYPE_K){for(var m=0,c=n.length;c>m;m++)if(n[m]==(d.xAxisIndex||0)){o.push(l);break}for(var m=0,c=a.length;c>m;m++)if(a[m]==(d.yAxisIndex||0)){o.push(l);break}null==this.zoomOption.xAxisIndex&&null==this.zoomOption.yAxisIndex&&d.data&&d.data[0]&&d.data[0]instanceof Array&&(d.type==r.CHART_TYPE_SCATTER||d.type==r.CHART_TYPE_LINE||d.type==r.CHART_TYPE_BAR)&&o.push(l)}var p=null!=this._zoom.start?this._zoom.start:null!=this.zoomOption.start?this.zoomOption.start:0,u=null!=this._zoom.end?this._zoom.end:null!=this.zoomOption.end?this.zoomOption.end:100;p>u&&(p+=u,u=p-u,p-=u);var V=Math.round((u-p)/100*("horizontal"==this.zoomOption.orient?this._location.width:this._location.height));return{start:p,end:u,start2:0,end2:100,size:V,xAxisIndex:n,yAxisIndex:a,seriesIndex:o,scatterMap:this._zoom.scatterMap||{}}},_backupData:function(){this._originalData={xAxis:{},yAxis:{},series:{}};for(var e=this.option.xAxis,t=this._zoom.xAxisIndex,i=0,n=t.length;n>i;i++)this._originalData.xAxis[t[i]]=e[t[i]].data;for(var a=this.option.yAxis,o=this._zoom.yAxisIndex,i=0,n=o.length;n>i;i++)this._originalData.yAxis[o[i]]=a[o[i]].data;for(var s,l=this.option.series,h=this._zoom.seriesIndex,i=0,n=h.length;n>i;i++)s=l[h[i]],this._originalData.series[h[i]]=s.data,s.data&&s.data[0]&&s.data[0]instanceof Array&&(s.type==r.CHART_TYPE_SCATTER||s.type==r.CHART_TYPE_LINE||s.type==r.CHART_TYPE_BAR)&&(this._backupScale(),this._calculScatterMap(h[i]))},_calculScatterMap:function(t){this._zoom.scatterMap=this._zoom.scatterMap||{},this._zoom.scatterMap[t]=this._zoom.scatterMap[t]||{};var i=e("../component"),n=i.get("axis"),a=l.clone(this.option.xAxis);"category"==a[0].type&&(a[0].type="value"),a[1]&&"category"==a[1].type&&(a[1].type="value");var o=new n(this.ecTheme,null,!1,{xAxis:a,series:this.option.series},this,"xAxis"),r=this.option.series[t].xAxisIndex||0;this._zoom.scatterMap[t].x=o.getAxis(r).getExtremum(),o.dispose(),a=l.clone(this.option.yAxis),"category"==a[0].type&&(a[0].type="value"),a[1]&&"category"==a[1].type&&(a[1].type="value"),o=new n(this.ecTheme,null,!1,{yAxis:a,series:this.option.series},this,"yAxis"),r=this.option.series[t].yAxisIndex||0,this._zoom.scatterMap[t].y=o.getAxis(r).getExtremum(),o.dispose()},_buildBackground:function(){var e=this._location.width,t=this._location.height;this.shapeList.push(new n({zlevel:this._zlevelBase,hoverable:!1,style:{x:this._location.x,y:this._location.y,width:e,height:t,color:this.zoomOption.backgroundColor}}));for(var i=0,o=this._originalData.xAxis,s=this._zoom.xAxisIndex,l=0,h=s.length;h>l;l++)i=Math.max(i,o[s[l]].length);for(var d=this._originalData.yAxis,m=this._zoom.yAxisIndex,l=0,h=m.length;h>l;l++)i=Math.max(i,d[m[l]].length);for(var c,p=this._zoom.seriesIndex[0],u=this._originalData.series[p],V=Number.MIN_VALUE,U=Number.MAX_VALUE,l=0,h=u.length;h>l;l++)c=null!=u[l]?null!=u[l].value?u[l].value:u[l]:0,this.option.series[p].type==r.CHART_TYPE_K&&(c=c[1]),isNaN(c)&&(c=0),V=Math.max(V,c),U=Math.min(U,c);var y=V-U,g=[],f=e/(i-(i>1?1:0)),b=t/(i-(i>1?1:0)),_=1;"horizontal"==this.zoomOption.orient&&1>f?_=Math.floor(3*i/e):"vertical"==this.zoomOption.orient&&1>b&&(_=Math.floor(3*i/t));for(var l=0,h=i;h>l;l+=_)c=null!=u[l]?null!=u[l].value?u[l].value:u[l]:0,this.option.series[p].type==r.CHART_TYPE_K&&(c=c[1]),isNaN(c)&&(c=0),g.push("horizontal"==this.zoomOption.orient?[this._location.x+f*l,this._location.y+t-1-Math.round((c-U)/y*(t-10))]:[this._location.x+1+Math.round((c-U)/y*(e-10)),this._location.y+b*l]);"horizontal"==this.zoomOption.orient?(g.push([this._location.x+e,this._location.y+t]),g.push([this._location.x,this._location.y+t])):(g.push([this._location.x,this._location.y+t]),g.push([this._location.x,this._location.y])),this.shapeList.push(new a({zlevel:this._zlevelBase,style:{pointList:g,color:this.zoomOption.dataBackgroundColor},hoverable:!1}))},_buildFiller:function(){this._fillerShae={zlevel:this._zlevelBase,draggable:!0,ondrift:this._ondrift,ondragend:this._ondragend,_type:"filler"},this._fillerShae.style="horizontal"==this.zoomOption.orient?{x:this._location.x+Math.round(this._zoom.start/100*this._location.width)+this._handleSize,y:this._location.y,width:this._zoom.size-2*this._handleSize,height:this._location.height,color:this.zoomOption.fillerColor,text:":::",textPosition:"inside"}:{x:this._location.x,y:this._location.y+Math.round(this._zoom.start/100*this._location.height)+this._handleSize,width:this._location.width,height:this._zoom.size-2*this._handleSize,color:this.zoomOption.fillerColor,text:"::",textPosition:"inside"},this._fillerShae.highlightStyle={brushType:"fill",color:"rgba(0,0,0,0)"},this._fillerShae=new n(this._fillerShae),this.shapeList.push(this._fillerShae)},_buildHandle:function(){this._startShape={zlevel:this._zlevelBase,draggable:!0,style:{iconType:"rectangle",x:this._location.x,y:this._location.y,width:this._handleSize,height:this._handleSize,color:this.zoomOption.handleColor,text:"=",textPosition:"inside"},highlightStyle:{text:"",brushType:"fill",textPosition:"left"},ondrift:this._ondrift,ondragend:this._ondragend},"horizontal"==this.zoomOption.orient?(this._startShape.style.height=this._location.height,this._endShape=l.clone(this._startShape),this._startShape.style.x=this._fillerShae.style.x-this._handleSize,this._endShape.style.x=this._fillerShae.style.x+this._fillerShae.style.width,this._endShape.highlightStyle.textPosition="right"):(this._startShape.style.width=this._location.width,this._endShape=l.clone(this._startShape),this._startShape.style.y=this._fillerShae.style.y-this._handleSize,this._startShape.highlightStyle.textPosition="top",this._endShape.style.y=this._fillerShae.style.y+this._fillerShae.style.height,this._endShape.highlightStyle.textPosition="bottom"),this._startShape=new o(this._startShape),this._endShape=new o(this._endShape),this.shapeList.push(this._startShape),this.shapeList.push(this._endShape)},_buildFrame:function(){var e=this.subPixelOptimize(this._location.x,1),t=this.subPixelOptimize(this._location.y,1);this._startFrameShape={zlevel:this._zlevelBase,hoverable:!1,style:{x:e,y:t,width:this._location.width-(e>this._location.x?1:0),height:this._location.height-(t>this._location.y?1:0),lineWidth:1,brushType:"stroke",strokeColor:this.zoomOption.handleColor}},this._endFrameShape=l.clone(this._startFrameShape),this._startFrameShape=new n(this._startFrameShape),this._endFrameShape=new n(this._endFrameShape),this.shapeList.push(this._startFrameShape),this.shapeList.push(this._endFrameShape)},_syncHandleShape:function(){"horizontal"==this.zoomOption.orient?(this._startShape.style.x=this._fillerShae.style.x-this._handleSize,this._endShape.style.x=this._fillerShae.style.x+this._fillerShae.style.width,this._zoom.start=Math.floor((this._startShape.style.x-this._location.x)/this._location.width*100),this._zoom.end=Math.ceil((this._endShape.style.x+this._handleSize-this._location.x)/this._location.width*100)):(this._startShape.style.y=this._fillerShae.style.y-this._handleSize,this._endShape.style.y=this._fillerShae.style.y+this._fillerShae.style.height,this._zoom.start=Math.floor((this._startShape.style.y-this._location.y)/this._location.height*100),this._zoom.end=Math.ceil((this._endShape.style.y+this._handleSize-this._location.y)/this._location.height*100)),this.zr.modShape(this._startShape.id),this.zr.modShape(this._endShape.id),this._syncFrameShape(),this.zr.refresh()},_syncFillerShape:function(){var e,t;"horizontal"==this.zoomOption.orient?(e=this._startShape.style.x,t=this._endShape.style.x,this._fillerShae.style.x=Math.min(e,t)+this._handleSize,this._fillerShae.style.width=Math.abs(e-t)-this._handleSize,this._zoom.start=Math.floor((Math.min(e,t)-this._location.x)/this._location.width*100),this._zoom.end=Math.ceil((Math.max(e,t)+this._handleSize-this._location.x)/this._location.width*100)):(e=this._startShape.style.y,t=this._endShape.style.y,this._fillerShae.style.y=Math.min(e,t)+this._handleSize,this._fillerShae.style.height=Math.abs(e-t)-this._handleSize,this._zoom.start=Math.floor((Math.min(e,t)-this._location.y)/this._location.height*100),this._zoom.end=Math.ceil((Math.max(e,t)+this._handleSize-this._location.y)/this._location.height*100)),this.zr.modShape(this._fillerShae.id),this._syncFrameShape(),this.zr.refresh()},_syncFrameShape:function(){"horizontal"==this.zoomOption.orient?(this._startFrameShape.style.width=this._fillerShae.style.x-this._location.x,this._endFrameShape.style.x=this._fillerShae.style.x+this._fillerShae.style.width,this._endFrameShape.style.width=this._location.x+this._location.width-this._endFrameShape.style.x):(this._startFrameShape.style.height=this._fillerShae.style.y-this._location.y,this._endFrameShape.style.y=this._fillerShae.style.y+this._fillerShae.style.height,this._endFrameShape.style.height=this._location.y+this._location.height-this._endFrameShape.style.y),this.zr.modShape(this._startFrameShape.id),this.zr.modShape(this._endFrameShape.id)},_syncShape:function(){this.zoomOption.show&&("horizontal"==this.zoomOption.orient?(this._startShape.style.x=this._location.x+this._zoom.start/100*this._location.width,this._endShape.style.x=this._location.x+this._zoom.end/100*this._location.width-this._handleSize,this._fillerShae.style.x=this._startShape.style.x+this._handleSize,this._fillerShae.style.width=this._endShape.style.x-this._startShape.style.x-this._handleSize):(this._startShape.style.y=this._location.y+this._zoom.start/100*this._location.height,this._endShape.style.y=this._location.y+this._zoom.end/100*this._location.height-this._handleSize,this._fillerShae.style.y=this._startShape.style.y+this._handleSize,this._fillerShae.style.height=this._endShape.style.y-this._startShape.style.y-this._handleSize),this.zr.modShape(this._startShape.id),this.zr.modShape(this._endShape.id),this.zr.modShape(this._fillerShae.id),this._syncFrameShape(),this.zr.refresh())},_syncData:function(e){var t,i,n,a,o;for(var s in this._originalData){t=this._originalData[s];for(var l in t)o=t[l],null!=o&&(a=o.length,i=Math.floor(this._zoom.start/100*a),n=Math.ceil(this._zoom.end/100*a),this.option[s][l].data[0]instanceof Array&&this.option[s][l].type!=r.CHART_TYPE_K?(this._setScale(),this.option[s][l].data=this._synScatterData(l,o)):this.option[s][l].data=o.slice(i,n))}this._isSilence||!this.zoomOption.realtime&&!e||this.messageCenter.dispatch(r.EVENT.DATA_ZOOM,null,{zoom:this._zoom},this.myChart)},_synScatterData:function(e,t){if(0===this._zoom.start&&100==this._zoom.end&&0===this._zoom.start2&&100==this._zoom.end2)return t;var i,n,a,o,r,s=[],l=this._zoom.scatterMap[e];"horizontal"==this.zoomOption.orient?(i=l.x.max-l.x.min,n=this._zoom.start/100*i+l.x.min,a=this._zoom.end/100*i+l.x.min,i=l.y.max-l.y.min,o=this._zoom.start2/100*i+l.y.min,r=this._zoom.end2/100*i+l.y.min):(i=l.x.max-l.x.min,n=this._zoom.start2/100*i+l.x.min,a=this._zoom.end2/100*i+l.x.min,i=l.y.max-l.y.min,o=this._zoom.start/100*i+l.y.min,r=this._zoom.end/100*i+l.y.min);for(var h,d=0,m=t.length;m>d;d++)h=t[d].value||t[d],h[0]>=n&&h[0]<=a&&h[1]>=o&&h[1]<=r&&s.push(t[d]);return s},_setScale:function(){var e=0!==this._zoom.start||100!==this._zoom.end||0!==this._zoom.start2||100!==this._zoom.end2,t={xAxis:this.option.xAxis,yAxis:this.option.yAxis};for(var i in t)for(var n=0,a=t[i].length;a>n;n++)t[i][n].scale=e||t[i][n]._scale},_backupScale:function(){var e={xAxis:this.option.xAxis,yAxis:this.option.yAxis};for(var t in e)for(var i=0,n=e[t].length;n>i;i++)e[t][i]._scale=e[t][i].scale},_getDetail:function(){var e="horizontal"==this.zoomOption.orient?"xAxis":"yAxis",t=this._originalData[e];for(var i in t){var n=t[i];if(null!=n){var a=n.length,o=Math.floor(this._zoom.start/100*a),r=Math.ceil(this._zoom.end/100*a);return r-=r>=a?1:0,{start:null!=n[o].value?n[o].value:n[o],end:null!=n[r].value?n[r].value:n[r]}}}var l=this._zoom.seriesIndex[0],h=this.option.series[l][e+"Index"]||0,d=this.option[e][h].type,m=this._zoom.scatterMap[l][e.charAt(0)].min,c=this._zoom.scatterMap[l][e.charAt(0)].max,p=c-m;if("value"==d)return{start:m+p*this._zoom.start/100,end:m+p*this._zoom.end/100};if("time"==d){c=m+p*this._zoom.end/100,m+=p*this._zoom.start/100;var u=s.getAutoFormatter(m,c).formatter;return{start:s.format(u,m),end:s.format(u,c)}}return{start:"",end:""}},__ondrift:function(e,t,i){this.zoomOption.zoomLock&&(e=this._fillerShae);var n="filler"==e._type?this._handleSize:0;if("horizontal"==this.zoomOption.orient?e.style.x+t-n<=this._location.x?e.style.x=this._location.x+n:e.style.x+t+e.style.width+n>=this._location.x+this._location.width?e.style.x=this._location.x+this._location.width-e.style.width-n:e.style.x+=t:e.style.y+i-n<=this._location.y?e.style.y=this._location.y+n:e.style.y+i+e.style.height+n>=this._location.y+this._location.height?e.style.y=this._location.y+this._location.height-e.style.height-n:e.style.y+=i,"filler"==e._type?this._syncHandleShape():this._syncFillerShape(),this.zoomOption.realtime&&this._syncData(),this.zoomOption.showDetail){var a=this._getDetail();this._startShape.style.text=this._startShape.highlightStyle.text=a.start,this._endShape.style.text=this._endShape.highlightStyle.text=a.end,this._startShape.style.textPosition=this._startShape.highlightStyle.textPosition,this._endShape.style.textPosition=this._endShape.highlightStyle.textPosition}return!0},__ondragend:function(){this.zoomOption.showDetail&&(this._startShape.style.text=this._endShape.style.text="=",this._startShape.style.textPosition=this._endShape.style.textPosition="inside",this.zr.modShape(this._startShape.id),this.zr.modShape(this._endShape.id),this.zr.refreshNextFrame()),this.isDragend=!0},ondragend:function(e,t){this.isDragend&&e.target&&(!this.zoomOption.realtime&&this._syncData(),t.dragOut=!0,t.dragIn=!0,this._isSilence||this.zoomOption.realtime||this.messageCenter.dispatch(r.EVENT.DATA_ZOOM,null,{zoom:this._zoom},this.myChart),t.needRefresh=!1,this.isDragend=!1)},ondataZoom:function(e,t){t.needRefresh=!0},absoluteZoom:function(e){this._zoom.start=e.start,this._zoom.end=e.end,this._zoom.start2=e.start2,this._zoom.end2=e.end2,this._syncShape(),this._syncData(!0)},rectZoom:function(e){if(!e)return this._zoom.start=this._zoom.start2=0,this._zoom.end=this._zoom.end2=100,this._syncShape(),this._syncData(!0),this._zoom;var t=this.component.grid.getArea(),i={x:e.x,y:e.y,width:e.width,height:e.height};if(i.width<0&&(i.x+=i.width,i.width=-i.width),i.height<0&&(i.y+=i.height,i.height=-i.height),i.x>t.x+t.width||i.y>t.y+t.height)return!1;i.xt.x+t.width&&(i.width=t.x+t.width-i.x),i.y+i.height>t.y+t.height&&(i.height=t.y+t.height-i.y);var n,a=(i.x-t.x)/t.width,o=1-(i.x+i.width-t.x)/t.width,r=1-(i.y+i.height-t.y)/t.height,s=(i.y-t.y)/t.height;return"horizontal"==this.zoomOption.orient?(n=this._zoom.end-this._zoom.start,this._zoom.start+=n*a,this._zoom.end-=n*o,n=this._zoom.end2-this._zoom.start2,this._zoom.start2+=n*r,this._zoom.end2-=n*s):(n=this._zoom.end-this._zoom.start,this._zoom.start+=n*r,this._zoom.end-=n*s,n=this._zoom.end2-this._zoom.start2,this._zoom.start2+=n*a,this._zoom.end2-=n*o),this._syncShape(),this._syncData(!0),this._zoom},syncBackupData:function(e){for(var t,i,n=this._originalData.series,a=e.series,o=0,r=a.length;r>o;o++){i=a[o].data||a[o].eventList,t=n[o]?Math.floor(this._zoom.start/100*n[o].length):0;for(var s=0,l=i.length;l>s;s++)n[o]&&(n[o][s+t]=i[s])}},syncOption:function(e){this.silence(!0),this.option=e,this.option.dataZoom=this.reformOption(this.option.dataZoom),this.zoomOption=this.option.dataZoom,this.myChart.canvasSupported||(this.zoomOption.realtime=!1),this.clear(),this._location=this._getLocation(),this._zoom=this._getZoom(),this._backupData(),this.option.dataZoom&&this.option.dataZoom.show&&this._buildShape(),this._syncData(),this.silence(!1)},silence:function(e){this._isSilence=e},getRealDataIndex:function(e,t){if(!this._originalData||0===this._zoom.start&&100==this._zoom.end)return t;var i=this._originalData.series;return i[e]?Math.floor(this._zoom.start/100*i[e].length)+t:-1},resize:function(){this.clear(),this._location=this._getLocation(),this._zoom=this._getZoom(),this.option.dataZoom.show&&this._buildShape()}},l.inherits(t,i),e("../component").define("dataZoom",t),t}),define("echarts/component/categoryAxis",["require","./base","zrender/shape/Text","zrender/shape/Line","zrender/shape/Rectangle","../config","zrender/tool/util","zrender/tool/area","../component"],function(e){function t(e,t,n,a,o,r){if(a.data.length<1)return void console.error("option.data.length < 1.");i.call(this,e,t,n,a,o),this.grid=this.component.grid;for(var s in r)this[s]=r[s];this.refresh(a)}var i=e("./base"),n=e("zrender/shape/Text"),a=e("zrender/shape/Line"),o=e("zrender/shape/Rectangle"),r=e("../config"),s=e("zrender/tool/util"),l=e("zrender/tool/area");return t.prototype={type:r.COMPONENT_TYPE_AXIS_CATEGORY,_getReformedLabel:function(e){var t="undefined"!=typeof this.option.data[e].value?this.option.data[e].value:this.option.data[e],i=this.option.data[e].formatter||this.option.axisLabel.formatter;return i&&("function"==typeof i?t=i.call(this.myChart,t):"string"==typeof i&&(t=i.replace("{value}",t))),t},_getInterval:function(){var e=this.option.axisLabel.interval;if("auto"==e){var t=this.option.axisLabel.textStyle.fontSize,i=this.option.data,n=this.option.data.length;if(this.isHorizontal())if(n>3){var a,o,r=this.getGap(),h=!1,d=Math.floor(.5/r);for(d=1>d?1:d,e=Math.floor(15/r);!h&&n>e;){e+=d,h=!0,a=Math.floor(r*e);for(var m=Math.floor((n-1)/e)*e;m>=0;m-=e){if(0!==this.option.axisLabel.rotate)o=t;else if(i[m].textStyle)o=l.getTextWidth(this._getReformedLabel(m),this.getFont(s.merge(i[m].textStyle,this.option.axisLabel.textStyle)));else{var c=this._getReformedLabel(m)+"",p=(c.match(/\w/g)||"").length,u=c.length-p;o=p*t*2/3+u*t}if(o>a){h=!1;break}}}}else e=1;else if(n>3){var r=this.getGap();for(e=Math.floor(11/r);t>r*e-6&&n>e;)e++}else e=1}else e=e-0+1;return e},_buildShape:function(){if(this._interval=this._getInterval(),this.option.show){this.option.splitArea.show&&this._buildSplitArea(),this.option.splitLine.show&&this._buildSplitLine(),this.option.axisLine.show&&this._buildAxisLine(),this.option.axisTick.show&&this._buildAxisTick(),this.option.axisLabel.show&&this._buildAxisLabel();for(var e=0,t=this.shapeList.length;t>e;e++)this.zr.addShape(this.shapeList[e])}},_buildAxisTick:function(){var e,t=this.option.data.length,i=this.option.axisTick,n=i.length,o=i.lineStyle.color,r=i.lineStyle.width,s="auto"==i.interval?this._interval:i.interval-0+1,l=i.onGap,h=l?this.getGap()/2:"undefined"==typeof l&&this.option.boundaryGap?this.getGap()/2:0,d=h>0?-s:0;if(this.isHorizontal())for(var m,c="bottom"==this.option.position?i.inside?this.grid.getYend()-n-1:this.grid.getYend()+1:i.inside?this.grid.getY()+1:this.grid.getY()-n-1,p=d;t>p;p+=s)m=this.subPixelOptimize(this.getCoordByIndex(p)+(p>=0?h:0),r),e={_axisShape:"axisTick",zlevel:this._zlevelBase,hoverable:!1,style:{xStart:m,yStart:c,xEnd:m,yEnd:c+n,strokeColor:o,lineWidth:r}},this.shapeList.push(new a(e)); +else for(var u,V="left"==this.option.position?i.inside?this.grid.getX()+1:this.grid.getX()-n-1:i.inside?this.grid.getXend()-n-1:this.grid.getXend()+1,p=d;t>p;p+=s)u=this.subPixelOptimize(this.getCoordByIndex(p)-(p>=0?h:0),r),e={_axisShape:"axisTick",zlevel:this._zlevelBase,hoverable:!1,style:{xStart:V,yStart:u,xEnd:V+n,yEnd:u,strokeColor:o,lineWidth:r}},this.shapeList.push(new a(e))},_buildAxisLabel:function(){var e,t,i=this.option.data,a=this.option.data.length,o=this.option.axisLabel.rotate,r=this.option.axisLabel.margin,l=this.option.axisLabel.clickable,h=this.option.axisLabel.textStyle;if(this.isHorizontal()){var d,m;"bottom"==this.option.position?(d=this.grid.getYend()+r,m="top"):(d=this.grid.getY()-r,m="bottom");for(var c=0;a>c;c+=this._interval)""!==this._getReformedLabel(c)&&(t=s.merge(i[c].textStyle||{},h),e={zlevel:this._zlevelBase,hoverable:!1,style:{x:this.getCoordByIndex(c),y:d,color:t.color,text:this._getReformedLabel(c),textFont:this.getFont(t),textAlign:t.align||"center",textBaseline:t.baseline||m}},o&&(e.style.textAlign=o>0?"bottom"==this.option.position?"right":"left":"bottom"==this.option.position?"left":"right",e.rotation=[o*Math.PI/180,e.style.x,e.style.y]),this.shapeList.push(new n(this._axisLabelClickable(l,e))))}else{var p,u;"left"==this.option.position?(p=this.grid.getX()-r,u="right"):(p=this.grid.getXend()+r,u="left");for(var c=0;a>c;c+=this._interval)""!==this._getReformedLabel(c)&&(t=s.merge(i[c].textStyle||{},h),e={zlevel:this._zlevelBase,hoverable:!1,style:{x:p,y:this.getCoordByIndex(c),color:t.color,text:this._getReformedLabel(c),textFont:this.getFont(t),textAlign:t.align||u,textBaseline:t.baseline||0===c&&""!==this.option.name?"bottom":c==a-1&&""!==this.option.name?"top":"middle"}},o&&(e.rotation=[o*Math.PI/180,e.style.x,e.style.y]),this.shapeList.push(new n(this._axisLabelClickable(l,e))))}},_buildSplitLine:function(){var e,t=this.option.data.length,i=this.option.splitLine,n=i.lineStyle.type,o=i.lineStyle.width,r=i.lineStyle.color;r=r instanceof Array?r:[r];var s=r.length,l=i.onGap,h=l?this.getGap()/2:"undefined"==typeof l&&this.option.boundaryGap?this.getGap()/2:0;if(t-=l||"undefined"==typeof l&&this.option.boundaryGap?1:0,this.isHorizontal())for(var d,m=this.grid.getY(),c=this.grid.getYend(),p=0;t>p;p+=this._interval)d=this.subPixelOptimize(this.getCoordByIndex(p)+h,o),e={zlevel:this._zlevelBase,hoverable:!1,style:{xStart:d,yStart:m,xEnd:d,yEnd:c,strokeColor:r[p/this._interval%s],lineType:n,lineWidth:o}},this.shapeList.push(new a(e));else for(var u,V=this.grid.getX(),U=this.grid.getXend(),p=0;t>p;p+=this._interval)u=this.subPixelOptimize(this.getCoordByIndex(p)-h,o),e={zlevel:this._zlevelBase,hoverable:!1,style:{xStart:V,yStart:u,xEnd:U,yEnd:u,strokeColor:r[p/this._interval%s],linetype:n,lineWidth:o}},this.shapeList.push(new a(e))},_buildSplitArea:function(){var e,t=this.option.splitArea,i=t.areaStyle.color;if(i instanceof Array){var n=i.length,a=this.option.data.length,r=t.onGap,s=r?this.getGap()/2:"undefined"==typeof r&&this.option.boundaryGap?this.getGap()/2:0;if(this.isHorizontal())for(var l,h=this.grid.getY(),d=this.grid.getHeight(),m=this.grid.getX(),c=0;a>=c;c+=this._interval)l=a>c?this.getCoordByIndex(c)+s:this.grid.getXend(),e={zlevel:this._zlevelBase,hoverable:!1,style:{x:m,y:h,width:l-m,height:d,color:i[c/this._interval%n]}},this.shapeList.push(new o(e)),m=l;else for(var p,u=this.grid.getX(),V=this.grid.getWidth(),U=this.grid.getYend(),c=0;a>=c;c+=this._interval)p=a>c?this.getCoordByIndex(c)-s:this.grid.getY(),e={zlevel:this._zlevelBase,hoverable:!1,style:{x:u,y:p,width:V,height:U-p,color:i[c/this._interval%n]}},this.shapeList.push(new o(e)),U=p}else e={zlevel:this._zlevelBase,hoverable:!1,style:{x:this.grid.getX(),y:this.grid.getY(),width:this.grid.getWidth(),height:this.grid.getHeight(),color:i}},this.shapeList.push(new o(e))},refresh:function(e){e&&(this.option=this.reformOption(e),this.option.axisLabel.textStyle=s.merge(this.option.axisLabel.textStyle||{},this.ecTheme.textStyle)),this.clear(),this._buildShape()},getGap:function(){var e=this.option.data.length,t=this.isHorizontal()?this.grid.getWidth():this.grid.getHeight();return this.option.boundaryGap?t/e:t/(e>1?e-1:1)},getCoord:function(e){for(var t=this.option.data,i=t.length,n=this.getGap(),a=this.option.boundaryGap?n/2:0,o=0;i>o;o++){if(t[o]==e||"undefined"!=typeof t[o].value&&t[o].value==e)return a=this.isHorizontal()?this.grid.getX()+a:this.grid.getYend()-a;a+=n}},getCoordByIndex:function(e){if(0>e)return this.isHorizontal()?this.grid.getX():this.grid.getYend();if(e>this.option.data.length-1)return this.isHorizontal()?this.grid.getXend():this.grid.getY();var t=this.getGap(),i=this.option.boundaryGap?t/2:0;return i+=e*t,i=this.isHorizontal()?this.grid.getX()+i:this.grid.getYend()-i},getNameByIndex:function(e){var t=this.option.data[e];return"undefined"!=typeof t&&"undefined"!=typeof t.value?t.value:t},getIndexByName:function(e){for(var t=this.option.data,i=t.length,n=0;i>n;n++)if(t[n]==e||"undefined"!=typeof t[n].value&&t[n].value==e)return n;return-1},getValueFromCoord:function(){return""},isMainAxis:function(e){return e%this._interval===0}},s.inherits(t,i),e("../component").define("categoryAxis",t),t}),define("echarts/component/valueAxis",["require","./base","zrender/shape/Text","zrender/shape/Line","zrender/shape/Rectangle","../config","../util/date","zrender/tool/util","../util/smartSteps","../util/accMath","../component"],function(e){function t(e,t,n,a,o,r,s){if(!s||0===s.length)return void console.err("option.series.length == 0.");i.call(this,e,t,n,a,o),this.series=s,this.grid=this.component.grid;for(var l in r)this[l]=r[l];this.refresh(a,s)}var i=e("./base"),n=e("zrender/shape/Text"),a=e("zrender/shape/Line"),o=e("zrender/shape/Rectangle"),r=e("../config"),s=e("../util/date"),l=e("zrender/tool/util");return t.prototype={type:r.COMPONENT_TYPE_AXIS_VALUE,_buildShape:function(){if(this._hasData=!1,this._calculateValue(),this._hasData&&this.option.show){this.option.splitArea.show&&this._buildSplitArea(),this.option.splitLine.show&&this._buildSplitLine(),this.option.axisLine.show&&this._buildAxisLine(),this.option.axisTick.show&&this._buildAxisTick(),this.option.axisLabel.show&&this._buildAxisLabel();for(var e=0,t=this.shapeList.length;t>e;e++)this.zr.addShape(this.shapeList[e])}},_buildAxisTick:function(){var e,t=this._valueList,i=this._valueList.length,n=this.option.axisTick,o=n.length,r=n.lineStyle.color,s=n.lineStyle.width;if(this.isHorizontal())for(var l,h="bottom"===this.option.position?n.inside?this.grid.getYend()-o-1:this.grid.getYend()+1:n.inside?this.grid.getY()+1:this.grid.getY()-o-1,d=0;i>d;d++)l=this.subPixelOptimize(this.getCoord(t[d]),s),e={_axisShape:"axisTick",zlevel:this._zlevelBase,hoverable:!1,style:{xStart:l,yStart:h,xEnd:l,yEnd:h+o,strokeColor:r,lineWidth:s}},this.shapeList.push(new a(e));else for(var m,c="left"===this.option.position?n.inside?this.grid.getX()+1:this.grid.getX()-o-1:n.inside?this.grid.getXend()-o-1:this.grid.getXend()+1,d=0;i>d;d++)m=this.subPixelOptimize(this.getCoord(t[d]),s),e={_axisShape:"axisTick",zlevel:this._zlevelBase,hoverable:!1,style:{xStart:c,yStart:m,xEnd:c+o,yEnd:m,strokeColor:r,lineWidth:s}},this.shapeList.push(new a(e))},_buildAxisLabel:function(){var e,t=this._valueList,i=this._valueList.length,a=this.option.axisLabel.rotate,o=this.option.axisLabel.margin,r=this.option.axisLabel.clickable,s=this.option.axisLabel.textStyle;if(this.isHorizontal()){var l,h;"bottom"===this.option.position?(l=this.grid.getYend()+o,h="top"):(l=this.grid.getY()-o,h="bottom");for(var d=0;i>d;d++)e={zlevel:this._zlevelBase,hoverable:!1,style:{x:this.getCoord(t[d]),y:l,color:"function"==typeof s.color?s.color(t[d]):s.color,text:this._valueLabel[d],textFont:this.getFont(s),textAlign:s.align||"center",textBaseline:s.baseline||h}},a&&(e.style.textAlign=a>0?"bottom"===this.option.position?"right":"left":"bottom"===this.option.position?"left":"right",e.rotation=[a*Math.PI/180,e.style.x,e.style.y]),this.shapeList.push(new n(this._axisLabelClickable(r,e)))}else{var m,c;"left"===this.option.position?(m=this.grid.getX()-o,c="right"):(m=this.grid.getXend()+o,c="left");for(var d=0;i>d;d++)e={zlevel:this._zlevelBase,hoverable:!1,style:{x:m,y:this.getCoord(t[d]),color:"function"==typeof s.color?s.color(t[d]):s.color,text:this._valueLabel[d],textFont:this.getFont(s),textAlign:s.align||c,textBaseline:s.baseline||0===d&&""!==this.option.name?"bottom":d===i-1&&""!==this.option.name?"top":"middle"}},a&&(e.rotation=[a*Math.PI/180,e.style.x,e.style.y]),this.shapeList.push(new n(this._axisLabelClickable(r,e)))}},_buildSplitLine:function(){var e,t=this._valueList,i=this._valueList.length,n=this.option.splitLine,o=n.lineStyle.type,r=n.lineStyle.width,s=n.lineStyle.color;s=s instanceof Array?s:[s];var l=s.length;if(this.isHorizontal())for(var h,d=this.grid.getY(),m=this.grid.getYend(),c=0;i>c;c++)h=this.subPixelOptimize(this.getCoord(t[c]),r),e={zlevel:this._zlevelBase,hoverable:!1,style:{xStart:h,yStart:d,xEnd:h,yEnd:m,strokeColor:s[c%l],lineType:o,lineWidth:r}},this.shapeList.push(new a(e));else for(var p,u=this.grid.getX(),V=this.grid.getXend(),c=0;i>c;c++)p=this.subPixelOptimize(this.getCoord(t[c]),r),e={zlevel:this._zlevelBase,hoverable:!1,style:{xStart:u,yStart:p,xEnd:V,yEnd:p,strokeColor:s[c%l],lineType:o,lineWidth:r}},this.shapeList.push(new a(e))},_buildSplitArea:function(){var e,t=this.option.splitArea.areaStyle.color;if(t instanceof Array){var i=t.length,n=this._valueList,a=this._valueList.length;if(this.isHorizontal())for(var r,s=this.grid.getY(),l=this.grid.getHeight(),h=this.grid.getX(),d=0;a>=d;d++)r=a>d?this.getCoord(n[d]):this.grid.getXend(),e={zlevel:this._zlevelBase,hoverable:!1,style:{x:h,y:s,width:r-h,height:l,color:t[d%i]}},this.shapeList.push(new o(e)),h=r;else for(var m,c=this.grid.getX(),p=this.grid.getWidth(),u=this.grid.getYend(),d=0;a>=d;d++)m=a>d?this.getCoord(n[d]):this.grid.getY(),e={zlevel:this._zlevelBase,hoverable:!1,style:{x:c,y:m,width:p,height:u-m,color:t[d%i]}},this.shapeList.push(new o(e)),u=m}else e={zlevel:this._zlevelBase,hoverable:!1,style:{x:this.grid.getX(),y:this.grid.getY(),width:this.grid.getWidth(),height:this.grid.getHeight(),color:t}},this.shapeList.push(new o(e))},_calculateValue:function(){if(isNaN(this.option.min-0)||isNaN(this.option.max-0)){for(var e,t,i={},n=this.component.legend,a=0,o=this.series.length;o>a;a++)!(this.series[a].type!=r.CHART_TYPE_LINE&&this.series[a].type!=r.CHART_TYPE_BAR&&this.series[a].type!=r.CHART_TYPE_SCATTER&&this.series[a].type!=r.CHART_TYPE_K&&this.series[a].type!=r.CHART_TYPE_EVENTRIVER||n&&!n.isSelected(this.series[a].name)||(e=this.series[a].xAxisIndex||0,t=this.series[a].yAxisIndex||0,this.option.xAxisIndex!=e&&this.option.yAxisIndex!=t||!this._calculSum(i,a)));var s;for(var a in i){s=i[a];for(var l=0,h=s.length;h>l;l++)if(!isNaN(s[l])){this._hasData=!0,this._min=s[l],this._max=s[l];break}if(this._hasData)break}for(var a in i){s=i[a];for(var l=0,h=s.length;h>l;l++)isNaN(s[l])||(this._min=Math.min(this._min,s[l]),this._max=Math.max(this._max,s[l]))}var d=Math.abs(this._max-this._min);this._min=isNaN(this.option.min-0)?this._min-Math.abs(d*this.option.boundaryGap[0]):this.option.min-0,this._max=isNaN(this.option.max-0)?this._max+Math.abs(d*this.option.boundaryGap[1]):this.option.max-0,this._min===this._max&&(0===this._max?this._max=1:this._max>0?this._min=this._max/this.option.splitNumber!=null?this.option.splitNumber:5:this._max=this._max/this.option.splitNumber!=null?this.option.splitNumber:5),"time"!=this.option.type?this._reformValue(this.option.scale):this._reformTimeValue()}else this._hasData=!0,this._min=this.option.min-0,this._max=this.option.max-0,"time"!=this.option.type?this._customerValue():this._reformTimeValue()},_calculSum:function(e,t){var i,n,a=this.series[t].name||"kener";if(this.series[t].stack){var o="__Magic_Key_Positive__"+this.series[t].stack,l="__Magic_Key_Negative__"+this.series[t].stack;e[o]=e[o]||[],e[l]=e[l]||[],e[a]=e[a]||[],n=this.series[t].data;for(var h=0,d=n.length;d>h;h++)i=null!=n[h].value?n[h].value:n[h],"-"!==i&&(i-=0,i>=0?null!=e[o][h]?e[o][h]+=i:e[o][h]=i:null!=e[l][h]?e[l][h]+=i:e[l][h]=i,this.option.scale&&e[a].push(i))}else if(e[a]=e[a]||[],this.series[t].type!=r.CHART_TYPE_EVENTRIVER){n=this.series[t].data;for(var h=0,d=n.length;d>h;h++)i=null!=n[h].value?n[h].value:n[h],this.series[t].type===r.CHART_TYPE_K?(e[a].push(i[0]),e[a].push(i[1]),e[a].push(i[2]),e[a].push(i[3])):i instanceof Array?(-1!=this.option.xAxisIndex&&e[a].push("time"!=this.option.type?i[0]:s.getNewDate(i[0])),-1!=this.option.yAxisIndex&&e[a].push("time"!=this.option.type?i[1]:s.getNewDate(i[1]))):e[a].push(i)}else{n=this.series[t].eventList;for(var h=0,d=n.length;d>h;h++)for(var m=n[h].evolution,c=0,p=m.length;p>c;c++)e[a].push(s.getNewDate(m[c].time))}},_reformValue:function(t){var i=e("../util/smartSteps"),n=this.option.splitNumber;!t&&this._min>=0&&this._max>=0&&(this._min=0),!t&&this._min<=0&&this._max<=0&&(this._max=0);var a=i(this._min,this._max,n);n=null!=n?n:a.secs,this.option.splitNumber=n,this._min=a.min,this._max=a.max,this._valueList=a.pnts,this._reformLabelData()},_reformTimeValue:function(){var e=null!=this.option.splitNumber?this.option.splitNumber:5,t=s.getAutoFormatter(this._min,this._max,e),i=t.formatter,n=t.gapValue;this._valueList=[s.getNewDate(this._min)];var a;switch(i){case"week":a=s.nextMonday(this._min);break;case"month":a=s.nextNthOnMonth(this._min,1);break;case"quarter":a=s.nextNthOnQuarterYear(this._min,1);break;case"half-year":a=s.nextNthOnHalfYear(this._min,1);break;case"year":a=s.nextNthOnYear(this._min,1);break;default:72e5>=n?a=(Math.floor(this._min/n)+1)*n:(a=s.getNewDate(this._min- -n),a.setHours(6*Math.round(a.getHours()/6)),a.setMinutes(0),a.setSeconds(0))}for(a-this._min=0&&(("month"==i||"quarter"==i||"half-year"==i||"year"==i)&&t.setDate(1),!(this._max-t=a;a++)this._valueList.push(t.accAdd(this._min,t.accMul(n,a)));this._reformLabelData()},_reformLabelData:function(e){this._valueLabel=[];var t=this.option.axisLabel.formatter;if(t)for(var i=0,n=this._valueList.length;n>i;i++)"function"==typeof t?this._valueLabel.push(e?t.call(this.myChart,this._valueList[i],e):t.call(this.myChart,this._valueList[i])):"string"==typeof t&&this._valueLabel.push(e?s.format(t,this._valueList[i]):t.replace("{value}",this._valueList[i]));else if(e)for(var i=0,n=this._valueList.length;n>i;i++)this._valueLabel.push(s.format(e,this._valueList[i]));else for(var i=0,n=this._valueList.length;n>i;i++)this._valueLabel.push(this.numAddCommas(this._valueList[i]))},getExtremum:function(){return this._calculateValue(),{min:this._min,max:this._max}},refresh:function(e,t){e&&(this.option=this.reformOption(e),this.option.axisLabel.textStyle=l.merge(this.option.axisLabel.textStyle||{},this.ecTheme.textStyle),this.series=t),this.zr&&(this.clear(),this._buildShape())},getCoord:function(e){e=ethis._max?this._max:e;var t;return t=this.isHorizontal()?this.grid.getX()+(e-this._min)/(this._max-this._min)*this.grid.getWidth():this.grid.getYend()-(e-this._min)/(this._max-this._min)*this.grid.getHeight()},getCoordSize:function(e){return Math.abs(this.isHorizontal()?e/(this._max-this._min)*this.grid.getWidth():e/(this._max-this._min)*this.grid.getHeight())},getValueFromCoord:function(e){var t;return this.isHorizontal()?(e=ethis.grid.getXend()?this.grid.getXend():e,t=this._min+(e-this.grid.getX())/this.grid.getWidth()*(this._max-this._min)):(e=ethis.grid.getYend()?this.grid.getYend():e,t=this._max-(e-this.grid.getY())/this.grid.getHeight()*(this._max-this._min)),t.toFixed(2)-0},isMaindAxis:function(e){for(var t=0,i=this._valueList.length;i>t;t++)if(this._valueList[t]===e)return!0;return!1}},l.inherits(t,i),e("../component").define("valueAxis",t),t}),define("echarts/util/date",[],function(){function e(e,t,i){i=i>1?i:2;for(var n,a,o,r,s=0,l=d.length;l>s;s++)if(n=d[s].value,a=Math.ceil(t/n)*n-Math.floor(e/n)*n,Math.round(a/n)<=1.2*i){o=d[s].formatter,r=d[s].value;break}return null==o&&(o="year",n=317088e5,a=Math.ceil(t/n)*n-Math.floor(e/n)*n,r=Math.round(a/(i-1)/n)*n),{formatter:o,gapValue:r}}function t(e){return 10>e?"0"+e:e}function i(e,i){("week"==e||"month"==e||"quarter"==e||"half-year"==e||"year"==e)&&(e="MM - dd\nyyyy");var n=h(i),a=n.getFullYear(),o=n.getMonth()+1,r=n.getDate(),s=n.getHours(),l=n.getMinutes(),d=n.getSeconds();return e=e.replace("MM",t(o)),e=e.toLowerCase(),e=e.replace("yyyy",a),e=e.replace("yy",a%100),e=e.replace("dd",t(r)),e=e.replace("d",r),e=e.replace("hh",t(s)),e=e.replace("h",s),e=e.replace("mm",t(l)),e=e.replace("m",l),e=e.replace("ss",t(d)),e=e.replace("s",d)}function n(e){return e=h(e),e.setDate(e.getDate()+8-e.getDay()),e}function a(e,t,i){return e=h(e),e.setMonth(Math.ceil((e.getMonth()+1)/i)*i),e.setDate(t),e}function o(e,t){return a(e,t,1)}function r(e,t){return a(e,t,3)}function s(e,t){return a(e,t,6)}function l(e,t){return a(e,t,12)}function h(e){return e instanceof Date?e:new Date("string"==typeof e?e.replace(/-/g,"/"):e)}var d=[{formatter:"hh : mm : ss",value:1e3},{formatter:"hh : mm : ss",value:5e3},{formatter:"hh : mm : ss",value:1e4},{formatter:"hh : mm : ss",value:15e3},{formatter:"hh : mm : ss",value:3e4},{formatter:"hh : mm\nMM - dd",value:6e4},{formatter:"hh : mm\nMM - dd",value:3e5},{formatter:"hh : mm\nMM - dd",value:6e5},{formatter:"hh : mm\nMM - dd",value:9e5},{formatter:"hh : mm\nMM - dd",value:18e5},{formatter:"hh : mm\nMM - dd",value:36e5},{formatter:"hh : mm\nMM - dd",value:72e5},{formatter:"hh : mm\nMM - dd",value:216e5},{formatter:"hh : mm\nMM - dd",value:432e5},{formatter:"MM - dd\nyyyy",value:864e5},{formatter:"week",value:6048e5},{formatter:"month",value:26784e5},{formatter:"quarter",value:8208e6},{formatter:"half-year",value:16416e6},{formatter:"year",value:32832e6}];return{getAutoFormatter:e,getNewDate:h,format:i,nextMonday:n,nextNthPerNmonth:a,nextNthOnMonth:o,nextNthOnQuarterYear:r,nextNthOnHalfYear:s,nextNthOnYear:l}}),define("echarts/util/smartSteps",[],function(){function e(e){return W.log(K(e))/W.LN10}function t(e){return W.pow(10,e)}function i(e){return e===X(e)}function n(e,t,n,a){f=a||{},b=f.steps||L,_=f.secs||v,n=w(+n||0)%99,e=+e||0,t=+t||0,x=k=0,"min"in f&&(e=+f.min||0,x=1),"max"in f&&(t=+f.max||0,k=1),e>t&&(t=[e,e=t][0]);var o=t-e;if(x&&k)return g(e,t,n);if((n||5)>o){if(i(e)&&i(t))return p(e,t,n);if(0===o)return u(e,t,n)}return h(e,t,n)}function a(e,i,n,a){a=a||0;var s=o((i-e)/n,-1),l=o(e,-1,1),h=o(i,-1),d=W.min(s.e,l.e,h.e);r(s,{c:0,e:d}),r(l,s,1),r(h,s),a+=d,e=l.c,i=h.c;for(var m=(i-e)/n,c=t(a),p=0,u=[],V=n+1;V--;)u[V]=(e+m*V)*c;if(0>a){p=U(c),m=+(m*c).toFixed(p),e=+(e*c).toFixed(p),i=+(i*c).toFixed(p);for(var V=u.length;V--;)u[V]=u[V].toFixed(p),0===+u[V]&&(u[V]="0")}else e*=c,i*=c,m*=c;return _=0,b=0,f=0,{min:e,max:i,secs:n,step:m,fix:p,exp:a,pnts:u}}function o(n,a,o){a=w(a%10)||2,0>a&&(i(n)?a=(""+K(n)).replace(/0+$/,"").length||1:(n=n.toFixed(15).replace(/0+$/,""),a=n.replace(".","").replace(/^[-0]+/,"").length,n=+n));var r=X(e(n))-a+1,s=+(n*t(-r)).toFixed(15)||0;return s=o?X(s):I(s),!s&&(r=0),(""+K(s)).length>a&&(r+=1,s/=10),{c:s,e:r}}function r(e,i,n){var a=i.e-e.e;a&&(e.e+=a,e.c*=t(-a),e.c=n?X(e.c):I(e.c))}function s(e,t,i){e.et[n];)n++;if(!t[n])for(i/=10,e.e+=1,n=0;i>t[n];)n++;return e.c=t[n],e}function h(e,t,n){var s,h=n||+_.slice(-1),u=l((t-e)/h,b),U=o(t-e),g=o(e,-1,1),f=o(t,-1);if(r(U,u),r(g,u,1),r(f,u),n?s=m(g,f,h):h=d(g,f),i(e)&&i(t)&&e*t>=0){if(h>t-e)return p(e,t,h);h=c(e,t,n,g,f,h)}var L=V(e,t,g.c,f.c);return g.c=L[0],f.c=L[1],(x||k)&&y(e,t,g,f),a(g.c,f.c,h,f.e)}function d(e,i){for(var n,a,o,r,s=[],h=_.length;h--;)n=_[h],a=l((i.c-e.c)/n,b),a=a.c*t(a.e),o=X(e.c/a)*a,r=I(i.c/a)*a,s[h]={min:o,max:r,step:a,span:r-o};return s.sort(function(e,t){return e.span-t.span}),s=s[0],n=s.span/s.step,e.c=s.min,i.c=s.max,3>n?2*n:n}function m(e,i,n){for(var a,o,r=i.c,s=(i.c-e.c)/n-1;r>e.c;)s=l(s+1,b),s=s.c*t(s.e),a=s*n,o=I(i.c/s)*s,r=o-a;var h=e.c-r,d=o-i.c,m=h-d;return m>=2*s&&(m=X(m/s)*s,r+=m,o+=m),e.c=r,i.c=o,s}function c(e,n,a,o,r,s){var l=r.c-o.c,h=l/s*t(r.e);if(!i(h)&&(h=X(h),l=h*s,n-e>l&&(h+=1,l=h*s,!a&&h*(s-1)>=n-e&&(s-=1,l=h*s)),l>=n-e)){var d=l-(n-e);o.c=w(e-d/2),r.c=w(n+d/2),o.e=0,r.e=0}return s}function p(e,t,i){if(i=i||5,x)t=e+i;else if(k)e=t-i;else{var n=i-(t-e),o=w(e-n/2),r=w(t+n/2),s=V(e,t,o,r);e=s[0],t=s[1]}return a(e,t,i)}function u(e,t,i){i=i||5;var n=W.min(K(t/i),i)/2.1;return x?t=e+n:k?e=t-n:(e-=n,t+=n),h(e,t,i)}function V(e,t,i,n){return e>=0&&0>i?(n-=i,i=0):0>=t&&n>0&&(i-=n,n=0),[i,n]}function U(e){return e=(+e).toFixed(15).split("."),e.pop().replace(/0+$/,"").length}function y(e,t,i,n){if(x){var a=o(e,4,1);i.e-a.e>6&&(a={c:0,e:i.e}),s(i,a),s(n,a),n.c+=a.c-i.c,i.c=a.c}else if(k){var r=o(t,4);n.e-r.e>6&&(r={c:0,e:n.e}),s(i,r),s(n,r),i.c+=r.c-n.c,n.c=r.c}}function g(e,t,i){var n=i?[i]:_,s=t-e;if(0===s)return t=o(t,3),i=n[0],t.c=w(t.c+i/2),a(t.c-i,t.c,i,t.e);K(t/s)<1e-6&&(t=0),K(e/s)<1e-6&&(e=0);var l,h,d,m=[[5,10],[10,2],[50,10],[100,2]],c=[],p=[],u=o(t-e,3),V=o(e,-1,1),U=o(t,-1);r(V,u,1),r(U,u),s=U.c-V.c,u.c=s;for(var y=n.length;y--;){i=n[y],l=I(s/i),h=l*i-s,d=3*(h+3),d+=2*(i-n[0]+2),i%5===0&&(d-=10);for(var g=m.length;g--;)l%m[g][0]===0&&(d/=m[g][1]);p[y]=[i,l,h,d].join(),c[y]={secs:i,step:l,delta:h,score:d}}return c.sort(function(e,t){return e.score-t.score}),c=c[0],V.c=w(V.c-c.delta/2),U.c=w(U.c+c.delta/2),a(V.c,U.c,c.secs,u.e)}var f,b,_,x,k,L=[10,25,50],v=[4,5,6],W=Math,w=W.round,X=W.floor,I=W.ceil,K=W.abs;return n}); \ No newline at end of file diff --git a/dev/cosbench-controller-web/resources/cosbench.css b/dev/cosbench-controller-web/resources/cosbench.css index e7fac098..36d84898 100644 --- a/dev/cosbench-controller-web/resources/cosbench.css +++ b/dev/cosbench-controller-web/resources/cosbench.css @@ -45,6 +45,7 @@ a:visited { background: transparent url('bg-header.png') repeat-x left top; height: 65px; position: fixed; + z-index: 2; top: 0px; left: 0px; width: 100%; @@ -373,3 +374,15 @@ a:visited { { background-color: #18A9FF; } +/* --------------------------------------- + * chart Styles + * --------------------------------------- */ + .chart{ + height: 400px; + width: 510px; + display: block; + float: left; + margin-left: 10px; + margin-top: 10px; + /*border: solid 1px #ccc;*/ + } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/AbstractClientHandler.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/AbstractClientHandler.java index 3c3cb647..cf494d1b 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/AbstractClientHandler.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/AbstractClientHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.handler; @@ -67,9 +67,9 @@ public ModelAndView handleRequest(HttpServletRequest req, } catch (ConfigException ce) { message = ce.getMessage(); } catch(FileNotFoundException fnfe) { - message = fnfe.getMessage(); + message = fnfe.getMessage(); } catch(IOException ie) { - message = ie.getMessage(); + message = ie.getMessage(); } catch (Exception e) { StringWriter writer = new StringWriter(); e.printStackTrace(new PrintWriter(writer)); diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/CancelHandler.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/CancelHandler.java index 70b7e715..6f7a5948 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/CancelHandler.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/CancelHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.handler; diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/ConfigHandler.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/ConfigHandler.java index 3cc58d5a..4063c002 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/ConfigHandler.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/ConfigHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.handler; diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/IndexHandler.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/IndexHandler.java index da3717a6..ed53f688 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/IndexHandler.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/IndexHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.handler; @@ -49,9 +49,9 @@ protected String process(HttpServletRequest req, HttpServletResponse res) .append(workload.getSubmitDate()).append('\t') .append(workload.getState()).append('\t'); if (workload.getCurrentStage() != null) { - buffer.append(workload.getCurrentStage().getId()).append('\n'); + buffer.append(workload.getCurrentStage().getId()).append('\n'); } else { - buffer.append("None").append('\n'); + buffer.append("None").append('\n'); } } buffer.append("Total:").append(' ').append(workloads.length) diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/SubmitHandler.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/SubmitHandler.java index 290751f5..4acf075e 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/SubmitHandler.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/SubmitHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.handler; diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/WorkloadHandler.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/WorkloadHandler.java index b31639db..a9bc8d4e 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/WorkloadHandler.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/handler/WorkloadHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.handler; diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/AdvancedConfigPageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/AdvancedConfigPageController.java index 6c7037d9..3e375898 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/AdvancedConfigPageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/AdvancedConfigPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -27,7 +27,7 @@ public class AdvancedConfigPageController extends AbstractController { @SuppressWarnings("unused") - private ControllerService controller; + private ControllerService controller; public void setController(ControllerService controller) { this.controller = controller; @@ -36,7 +36,9 @@ public void setController(ControllerService controller) { @Override protected ModelAndView process(HttpServletRequest req, HttpServletResponse res) { - return new ModelAndView("advanced-config"); + ModelAndView result = new ModelAndView("advanced-config"); + result.addObject("cInfo", controller.getControllerInfo()); + return result; } } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/CancelWorkloadController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/CancelWorkloadController.java index bf819615..bcbcedc6 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/CancelWorkloadController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/CancelWorkloadController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/CliLoginFilter.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/CliLoginFilter.java index 5ba4bfc1..79c9d3eb 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/CliLoginFilter.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/CliLoginFilter.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.web; import java.io.IOException; @@ -9,49 +27,49 @@ import com.intel.cosbench.utils.AuthValidator; public class CliLoginFilter implements javax.servlet.Filter { - protected ServletContext servletContext; - - private Map userInfo = new HashMap(); - public void init(FilterConfig filterConfig) { - servletContext = filterConfig.getServletContext(); - } - - public void doFilter(ServletRequest request, ServletResponse response, - FilterChain chain) throws IOException, ServletException { - - try { - if (!AuthValidator.NeedLogon(userInfo)) { - String username = (String) request.getParameter("username").toString(); - String password = (String) request.getParameter("password").toString(); - if (!username.isEmpty() && !password.isEmpty()) { - if (userInfo.containsKey(username) - && userInfo.get(username).equals(password)) { - chain.doFilter(request, response); - return; - } else { - response.getWriter().write( - "User info [username:password] incorrect !"); - return; - } - } else { - response.getWriter() - .write("Please input complete user info [username:password] to get authentication!"); - return; // need username and password - } - } else { - chain.doFilter(request, response); - } - } catch (Exception e) { - // TODO Auto-generated catch block - e.printStackTrace(); - } - - } - - @Override - public void destroy() { - // TODO Auto-generated method stub - - } + protected ServletContext servletContext; + + private Map userInfo = new HashMap(); + public void init(FilterConfig filterConfig) { + servletContext = filterConfig.getServletContext(); + } + + public void doFilter(ServletRequest request, ServletResponse response, + FilterChain chain) throws IOException, ServletException { + + try { + if (!AuthValidator.NeedLogon(userInfo)) { + String username = (String) request.getParameter("username").toString(); + String password = (String) request.getParameter("password").toString(); + if (!username.isEmpty() && !password.isEmpty()) { + if (userInfo.containsKey(username) + && userInfo.get(username).equals(password)) { + chain.doFilter(request, response); + return; + } else { + response.getWriter().write( + "User info [username:password] incorrect !"); + return; + } + } else { + response.getWriter() + .write("Please input complete user info [username:password] to get authentication!"); + return; // need username and password + } + } else { + chain.doFilter(request, response); + } + } catch (Exception e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + + } + + @Override + public void destroy() { + // TODO Auto-generated method stub + + } } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/ConfigPageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/ConfigPageController.java index 0c851576..4156269b 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/ConfigPageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/ConfigPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -27,7 +27,7 @@ public class ConfigPageController extends AbstractController { @SuppressWarnings("unused") - private ControllerService controller; + private ControllerService controller; public void setController(ControllerService controller) { this.controller = controller; @@ -36,7 +36,9 @@ public void setController(ControllerService controller) { @Override protected ModelAndView process(HttpServletRequest req, HttpServletResponse res) { - return new ModelAndView("config"); + ModelAndView result = new ModelAndView("config"); + result.addObject("cInfo", controller.getControllerInfo()); + return result; } } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/DownloadConfigController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/DownloadConfigController.java index 6cb028a1..94fcfd2f 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/DownloadConfigController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/DownloadConfigController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/DownloadLogController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/DownloadLogController.java index 6903248d..63deea06 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/DownloadLogController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/DownloadLogController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -57,7 +57,9 @@ public void render(Map model, HttpServletRequest req, protected ModelAndView process(WorkloadInfo info) { File file = controller.getWorkloadLog(info); - return new ModelAndView(LOG, "log", file); + ModelAndView result = new ModelAndView(LOG, "log", file); + result.addObject("cInfo", controller.getControllerInfo()); + return result; } } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/IndexPageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/IndexPageController.java index d9d0587a..6d20a0ff 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/IndexPageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/IndexPageController.java @@ -1,6 +1,7 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -12,8 +13,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -29,60 +30,60 @@ /** * The home page of controller web console. - * + * * @author ywang19, qzheng7 * */ public class IndexPageController extends AbstractController { - private ControllerService controller; + protected ControllerService controller; public void setController(ControllerService controller) { this.controller = controller; } - @Override - protected ModelAndView process(HttpServletRequest req, - HttpServletResponse res) throws IOException { - ModelAndView result = new ModelAndView("index"); - String id = req.getParameter("id"); - String up = req.getParameter("up"); - String neighId = req.getParameter("neighid"); - String cancelIds = req.getParameter("cancelIds"); - String cancel = req.getParameter("cancel"); - String resubmitIds = req.getParameter("resubmitIds"); - String resubmit = req.getParameter("resubmit"); - String loadArch = req.getParameter("loadArch"); - if (!StringUtils.isEmpty(id) && !StringUtils.isEmpty(up)) { - boolean isUp = up.equalsIgnoreCase("yes") ? true : false; - boolean changeOrderOk = controller.changeOrder(id, neighId, - isUp); - if (changeOrderOk) { - result.addObject("highlightId", id); - } - } else if (!StringUtils.isEmpty(cancelIds) - && !StringUtils.isEmpty(cancel)) { - String[] ids = cancelIds.split("_"); - for (String cancelId : ids) { - controller.cancel(cancelId); - } - } else if (!StringUtils.isEmpty(resubmit)&&!StringUtils.isEmpty(resubmitIds)){ - String[] ids = resubmitIds.split("_"); - for (String resubmitId : ids){ - String newId = controller.resubmit(resubmitId); - controller.fire(newId); - } - } - if (!StringUtils.isEmpty(loadArch) && loadArch.equals("true")) - controller.setloadArch(true); - else if (!StringUtils.isEmpty(loadArch) && loadArch.equals("false")) - controller.setloadArch(false); - + @Override + protected ModelAndView process(HttpServletRequest req, + HttpServletResponse res) throws IOException { + ModelAndView result = new ModelAndView("index"); + String id = req.getParameter("id"); + String up = req.getParameter("up"); + String neighId = req.getParameter("neighid"); + String cancelIds = req.getParameter("cancelIds"); + String cancel = req.getParameter("cancel"); + String resubmitIds = req.getParameter("resubmitIds"); + String resubmit = req.getParameter("resubmit"); + String loadArch = req.getParameter("loadArch"); + if (!StringUtils.isEmpty(id) && !StringUtils.isEmpty(up)) { + boolean isUp = up.equalsIgnoreCase("yes") ? true : false; + boolean changeOrderOk = controller.changeOrder(id, neighId, + isUp); + if (changeOrderOk) { + result.addObject("highlightId", id); + } + } else if (!StringUtils.isEmpty(cancelIds) + && !StringUtils.isEmpty(cancel)) { + String[] ids = cancelIds.split("_"); + for (String cancelId : ids) { + controller.cancel(cancelId); + } + } else if (!StringUtils.isEmpty(resubmit)&&!StringUtils.isEmpty(resubmitIds)){ + String[] ids = resubmitIds.split("_"); + for (String resubmitId : ids){ + String newId = controller.resubmit(resubmitId); + controller.fire(newId); + } + } + if (!StringUtils.isEmpty(loadArch) && loadArch.equals("true")) + controller.setloadArch(true); + else if (!StringUtils.isEmpty(loadArch) && loadArch.equals("false")) + controller.setloadArch(false); + result.addObject("cInfo", controller.getControllerInfo()); result.addObject("aInfos", controller.getActiveWorkloads()); result.addObject("hInfos", controller.getHistoryWorkloads()); result.addObject("archInfos", controller.getArchivedWorkloads()); result.addObject("loadArch", controller.getloadArch()); - return result; - } + return result; + } } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/LogonErrorPageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/LogonErrorPageController.java index f3f42dd2..1db5363d 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/LogonErrorPageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/LogonErrorPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -26,14 +26,14 @@ /** * The home page of controller web console. - * + * * @author ywang19, qzheng7 * */ public class LogonErrorPageController extends AbstractController { @SuppressWarnings("unused") - private ControllerService controller; + private ControllerService controller; public void setController(ControllerService controller) { this.controller = controller; @@ -43,6 +43,7 @@ public void setController(ControllerService controller) { protected ModelAndView process(HttpServletRequest req, HttpServletResponse res) { ModelAndView result = new ModelAndView("logonError"); + result.addObject("cInfo", controller.getControllerInfo()); return result; } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/LogonPageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/LogonPageController.java index 203e36b8..006c391c 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/LogonPageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/LogonPageController.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.web; import javax.servlet.http.HttpServletRequest; @@ -12,30 +30,31 @@ public class LogonPageController extends AbstractController { - @SuppressWarnings("unused") - private ControllerService controller; - - public void setController(ControllerService controller) { - this.controller = controller; - } - - @Override - protected ModelAndView process(HttpServletRequest req, - HttpServletResponse res) { - ModelAndView result = new ModelAndView("logon"); - try { - if(AuthValidator.NeedLogon()) { - result.addObject("username", AuthValidator.USERNAME); - result.addObject("password", AuthValidator.PASSWD); - result.addObject("hidden","visibility:hidden"); - } - else{ - result.addObject("username", ""); - result.addObject("password", ""); - } - } catch (Exception e) { - e.printStackTrace(); - } - return result; - } + @SuppressWarnings("unused") + private ControllerService controller; + + public void setController(ControllerService controller) { + this.controller = controller; + } + + @Override + protected ModelAndView process(HttpServletRequest req, + HttpServletResponse res) { + ModelAndView result = new ModelAndView("logon"); + try { + if(AuthValidator.NeedLogon()) { + result.addObject("username", AuthValidator.USERNAME); + result.addObject("password", AuthValidator.PASSWD); + result.addObject("hidden","visibility:hidden"); + } + else{ + result.addObject("username", ""); + result.addObject("password", ""); + } + } catch (Exception e) { + e.printStackTrace(); + } + result.addObject("cInfo", controller.getControllerInfo()); + return result; + } } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/MatrixPageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/MatrixPageController.java index ad739606..db82a5e3 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/MatrixPageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/MatrixPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -60,20 +60,21 @@ protected ModelAndView process(HttpServletRequest req, for (String other : others) result.addObject(other, true); if(req.getParameter("type").equals("histo")) - result.addObject("hInfos", controller.getHistoryWorkloads()); + result.addObject("hInfos", controller.getHistoryWorkloads()); else if (req.getParameter("type").equals("arch")) { - for (WorkloadInfo info : controller.getArchivedWorkloads()) { - if (info.getReport().getAllMetrics().length==0) { - try { - controller.getWorkloadLoader().loadWorkloadPageInfo(info); - } catch (IOException e) { - e.printStackTrace(); - } - } - } - result.addObject("hInfos", controller.getArchivedWorkloads()); + for (WorkloadInfo info : controller.getArchivedWorkloads()) { + if (info.getReport().getAllMetrics().length==0) { + try { + controller.getWorkloadLoader().loadWorkloadPageInfo(info); + } catch (IOException e) { + e.printStackTrace(); + } + } + } + result.addObject("hInfos", controller.getArchivedWorkloads()); } result.addObject("type", req.getParameter("type")); + result.addObject("cInfo", controller.getControllerInfo()); return result; } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/PrometheusController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/PrometheusController.java new file mode 100644 index 00000000..85c15e5e --- /dev/null +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/PrometheusController.java @@ -0,0 +1,324 @@ +/** + +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +/* + * + * PROMETHEUS EXPORTER + * + * @Author: Jérôme Loyet for OpenIO + */ +package com.intel.cosbench.controller.web; + +import java.io.IOException; +import java.io.PrintWriter; +import java.net.InetAddress; +import java.util.Date; +import java.util.HashMap; +import java.util.Iterator; +import java.util.Map; + +import javax.servlet.http.*; + +import org.apache.commons.lang.ArrayUtils; +import org.springframework.web.servlet.*; + +import com.intel.cosbench.bench.Metrics; +import com.intel.cosbench.bench.Report; +import com.intel.cosbench.model.*; +import com.intel.cosbench.service.ControllerService; + +public class PrometheusController extends IndexPageController { + + /* + * Custom "inline" View dedicated for Prometheus output generation + */ + private static final View PROMETHEUS = new PrometheusView(); + private static class PrometheusView implements View { + + /* + * # Prometheus FORMAT + * chart_name_with_underscores_separator{label1="value1",label2="value2"} 0.0000000 1546933475642 + * Followed by the value of the metric as a float + * At the end, the timestamp can be added (Unix time in milliseconds) + * + * More at https://prometheus.io/docs/instrumenting/exposition_formats/ + */ + + /* constants */ + private static final String PROM_SP = " "; + private static final String PROM_SEP = "_"; + private static final String PROM_LINE_SEP = "\n"; + private static final String PROM_LABEL_SEP = ","; + private static final String PROM_LABEL_QUOTE = "\""; + private static final String PROM_LABEL_EQ = "="; + private static final String PROM_LABEL_START = "{"; + private static final String PROM_LABEL_END = "}"; + private static final String PROM_PREFIX = "cosbench" + PROM_SEP; + + /* + * Output one metric with labels + */ + private void printProm(PrintWriter writer, long date, float value, String name, Map labels) { + + /* print chart name */ + writer.print(PROM_PREFIX + name); + + /* handle labels if present */ + if (labels != null) { + + Iterator> it = labels.entrySet().iterator(); + boolean first = true; + + /* if not labels list not empty */ + if (it.hasNext()) { + + writer.print(PROM_SP + PROM_LABEL_START); + + /* iterate through labels */ + while (it.hasNext()) { + + /* print PROM_LABL_SEP ',' only since the 2nd item */ + if (first) first = false; + else writer.print(PROM_LABEL_SEP); + + /* extract next label */ + Map.Entry label = (Map.Entry) it.next(); + + /* print the label */ + writer.print(label.getKey() + PROM_LABEL_EQ + PROM_LABEL_QUOTE + label.getValue() + PROM_LABEL_QUOTE); + } + + writer.print(PROM_LABEL_END); + } + } + + /* handle printing the float value to ensure it uses only num and the dot characters */ + if (value == (long) value) { + writer.print(PROM_SP + String.format("%d", (long)value)); + } else { + writer.print(PROM_SP + String.format("%s", value)); + } + + /* print the date if present */ + if (date > 0) writer.print(PROM_SP + String.valueOf(date)); + + /* End of the metric */ + writer.print(PROM_LINE_SEP); + } + + /* + * + */ + private void printMetrics(Report report, HashMap labels, PrintWriter writer) { + printMetrics(report, labels, 0, writer); + } + + /* + * Print complete operation metrics, composed of + * - Operations count + * - Transfer Byte count + * - Average Response time + * - Average Transfer time + * - Average Process time (which is Response time - Transfer Time) + * - Throughput (which is the number of operations per seconds) + * - Bandwidth (which is the number transfer bytes per seconds) + * - Success Percentage + */ + private void printMetrics(Report report, HashMap labels, long time, PrintWriter writer) { + /* duplicate labels for each workload */ + labels = (HashMap)labels.clone(); + + Metrics allMetrics[] = report.getAllMetrics(); + for (int i=0; i 0) opName = opId + ":" + opName; + labels.put("operation_name", opName); + + if (opType != null && opType.length() > 0) { + labels.put("operation_type", opType); + } + + labels.put("unit", "operation"); + printProm(writer, time, (float)metrics.getSampleCount(), "workload_ops_count", labels); + + labels.put("unit", "byte"); + printProm(writer, time, (float)metrics.getByteCount(), "workload_byte_count", labels); + + labels.put("unit", "ms"); + printProm(writer, time, (float)metrics.getAvgResTime(), "workload_average_response_time", labels); + + labels.put("unit", "ms"); + printProm(writer, time, (float)metrics.getAvgXferTime(), "workload_average_transfer_time", labels); + + labels.put("unit", "ms"); + printProm(writer, time, (float)metrics.getAvgResTime() - (long)metrics.getAvgXferTime(), "workload_average_process_time", labels); + + labels.put("unit", "op/s"); + printProm(writer, time, (float)metrics.getThroughput(), "workload_throughput", labels); + + labels.put("unit", "byte/s"); + printProm(writer, time, (float)metrics.getBandwidth(), "workload_bandwidth", labels); + + labels.put("unit", "%"); + printProm(writer, time, (float)(metrics.getRatio()*100), "workload_success_ratio", labels); + } + } + + /* + * Return the Content-Type for prometheus + * force charset to prevent tomcat to add another + */ + @Override + public String getContentType() { + return "text/plain; version=0.0.4; charset=utf-8"; + } + + /* + * Render the View + * Output statistics for Active and Historical Workloads by default + * Archive workloads are not exported by default + * change settings to : + * - prometheus.export.workloads.active + * - prometheus.export.workloads.historical + * - prometheus.export.workloads.archive + * + * By default statistics for loaded workloads are sent for 5 minutes after workload finished. + * The time can be changed with the setting 'prometheus.export.timeout'. + * Set it to 0 will disable this behavior and continue to send metrics until workload is loaded + * Warning: this can overlap with the parameter 'prometheus.export.workloads.archive' + */ + @Override + public void render(Map model, HttpServletRequest req, HttpServletResponse res) throws Exception { + + /* get the controller */ + ControllerService controller = (ControllerService)model.get("controller"); + + /* handle System properties */ + String includeActiveWorkloads = System.getProperty("prometheus.export.workloads.active"); + String includeHistoryWorkloads = System.getProperty("prometheus.export.workloads.history"); + String includeArchivedWorkloads = System.getProperty("prometheus.export.workloads.archived"); + int stopExporting; + try { + /* convert to int */ + stopExporting = Integer.parseInt(System.getProperty("prometheus.export.timeout")); + } catch (Exception e) { + /* default to 5 minutes*/ + stopExporting = 5; + } + + /* handle workloads to include */ + WorkloadInfo workloads[] = new WorkloadInfo[0]; + if (includeActiveWorkloads == null || "true".equalsIgnoreCase(includeActiveWorkloads)) workloads = (WorkloadInfo[])ArrayUtils.addAll(workloads, controller.getActiveWorkloads()); + if (includeHistoryWorkloads == null || "true".equalsIgnoreCase(includeHistoryWorkloads)) workloads = (WorkloadInfo[])ArrayUtils.addAll(workloads, controller.getHistoryWorkloads()); + if ("true".equalsIgnoreCase(includeArchivedWorkloads)) workloads = (WorkloadInfo[])ArrayUtils.addAll(workloads, controller.getArchivedWorkloads()); + + /* set Content-Type and get servlet Writer */ + res.setContentType(this.getContentType()); + PrintWriter writer = res.getWriter(); + + /* setup default labels with current hostname */ + HashMap labels = new HashMap(); + labels.put("instance", InetAddress.getLocalHost().getHostName()); + + /* iterate through each workloads */ + for (int i=0; i 0 && WorkloadState.isStopped(workload.getState())) { + if ((new Date()).getTime() > workload.getStopDate().getTime() + (1000 * 60 * stopExporting)) { + continue; + } + } + + /* set workload id and name as label by default */ + labels.put("workload", workload.getId() + " (" + workload.getWorkload().getName() + ")"); + + /* convert workload state to integer */ + float state; + switch(workload.getState()) { + case QUEUING: + state = 0; + break; + case PROCESSING: + state = 1; + break; + case FINISHED: + state = 2; + break; + case FAILED: + state = 3; + break; + case TERMINATED: + state = 4; + break; + case CANCELLED: + state = 5; + break; + default: + state = -1; + break; + } + + /* output workload_state metric */ + printProm(writer, 0, state, "workload_state", labels); + + // no other stats until it starts running + if (workload.getState() == WorkloadState.QUEUING) continue; + + /* get Report that contains all metrics + * If the workload is running (state == PROCESSING) then the report is to be fetch from the last SnapShot + * otherwise the workload is over and the report is to be taken directly + */ + Report report; + if (workload.getState() == WorkloadState.PROCESSING) { + report = workload.getSnapshot().getReport(); + } else { + report = workload.getReport(); + } + + /* Output all metrics */ + printMetrics(report, labels, writer); + + } + + /* print empty line */ + writer.print(PROM_LINE_SEP); + } + } + + /* + * Controller main function + */ + @Override + protected ModelAndView process(HttpServletRequest req, HttpServletResponse res) throws IOException { + + /* instanciate custom "inline" View */ + ModelAndView result = new ModelAndView(PROMETHEUS); + + /* pass controller Object to View */ + result.addObject("controller", controller); + + return result; + } +} diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/StagePageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/StagePageController.java index a6804fa3..dbfeb13c 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/StagePageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/StagePageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -57,18 +57,19 @@ protected ModelAndView process(String wid, String sid) { StageInfo sInfo = wInfo.getStageInfo(sid); if (sInfo == null) throw new NotFoundException(); - if (controller.getloadArch() && sInfo.getSnapshotRegistry().getSize() == 0) - try { - controller.getWorkloadLoader().loadStagePageInfo(wInfo, - sInfo.getId()); - } catch (IOException e) { - e.printStackTrace(); - } + if (controller.getloadArch() && sInfo.getSnapshotRegistry().getSize() == 0) + try { + controller.getWorkloadLoader().loadStagePageInfo(wInfo, + sInfo.getId()); + } catch (IOException e) { + e.printStackTrace(); + } return process(wInfo, sInfo); } protected ModelAndView process(WorkloadInfo wInfo, StageInfo sInfo) { ModelAndView result = new ModelAndView("stage"); + result.addObject("cInfo", controller.getControllerInfo()); result.addObject("sInfo", sInfo); result.addObject("wInfo", wInfo); result.addObject("isStopped", isStopped(sInfo.getState())); diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/SubmitPageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/SubmitPageController.java index 6d39ea89..3f235f8b 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/SubmitPageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/SubmitPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -37,7 +37,10 @@ public void setController(ControllerService controller) { protected ModelAndView process(HttpServletRequest req, HttpServletResponse res) { WorkloadInfo[] aInfos = controller.getActiveWorkloads(); - return new ModelAndView("submit", "aInfos", aInfos); + ModelAndView result = new ModelAndView("submit"); + result.addObject("aInfos", aInfos); + result.addObject("cInfo", controller.getControllerInfo()); + return result; } } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/TimelineCSVController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/TimelineCSVController.java index 8340662a..09046019 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/TimelineCSVController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/TimelineCSVController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -49,7 +49,9 @@ public void render(Map model, HttpServletRequest req, @Override protected ModelAndView process(WorkloadInfo wInfo, StageInfo sInfo) { - return new ModelAndView(CSV, "sInfo", sInfo); + ModelAndView result = new ModelAndView(CSV, "sInfo", sInfo); + result.addObject("cInfo", controller.getControllerInfo()); + return result; } } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/TimelinePageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/TimelinePageController.java index c4f3bf78..a32de90a 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/TimelinePageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/TimelinePageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -25,6 +25,7 @@ public class TimelinePageController extends StagePageController { protected ModelAndView process(WorkloadInfo wInfo, StageInfo sInfo) { ModelAndView result = new ModelAndView("timeline"); + result.addObject("cInfo", controller.getControllerInfo()); result.addObject("wInfo", wInfo); result.addObject("sInfo", sInfo); return result; diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadConfigGenerator.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadConfigGenerator.java index 054e3d27..b0f4b1a8 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadConfigGenerator.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadConfigGenerator.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.web; import java.io.ByteArrayInputStream; @@ -23,525 +41,525 @@ public class WorkloadConfigGenerator { - private String runtime; - private String delay; - private String rampup; - private String num_of_drivers; - private String auth_type; - private String auth_config; - private String storage_type; - private String storage_config; - private boolean generateWorkloadFiles; - private File WORKLOAD_CONFIG_DIR; - private static final String workloadConfigFilesRoot = "workloads"; - protected ControllerService controller; - - private static final Logger LOGGER = LogFactory.getSystemLogger(); - - - public WorkloadConfigGenerator(ControllerService controller) { - this.controller = controller; - } - - - public void createWorkloadFiles(HttpServletRequest req) throws Exception { - - // Set common workload parameters - setWorkloadParams(req); - - String workloadMatrixName = req.getParameter("workload.matrix.name"); - if(!workloadMatrixName.matches("[a-zA-Z0-9\\_\\-#\\.\\(\\)\\/%&]{3,50}")) - throw new Exception("Workload Matrix Name incorrect. Please use alphabets or numbers. Special characters allowed are _ - # . ( ) / % &. " - + "Length should be between 3 to 50 characters."); - String objectSizeStrings[] = req.getParameterValues("object-sizes"); - if (objectSizeStrings == null) - return; - - for (int i = 0; i < objectSizeStrings.length; i++) { - String workloadName = req.getParameterValues("workload.name")[i]; - if(!workloadName.matches("[a-zA-Z0-9\\_\\-#\\.\\(\\)\\/%&]{3,50}")) - throw new Exception("Workload Name incorrect. Please use alphabets or numbers. Special characters allowed are _ - # . ( ) / % &. " - + "Length should be between 3 to 50 characters."); - } - - String workloadNumbers[] = req.getParameterValues("workload-number"); - for (int i = 0; i < workloadNumbers.length; i++) { - String objectSizes[], unit; - boolean isRange; - - int workloadNumber = Integer.parseInt(workloadNumbers[i]); - String objectSizeString = objectSizeStrings[i]; - // if input is range of object sizes - if (objectSizeString.contains("-")) { - objectSizes = objectSizeString.split("-"); - unit = req.getParameterValues("object-size-unit")[i]; - isRange = true; - } - // if input is comma separated object size values - else { - objectSizes = objectSizeString.split(","); - unit = req.getParameterValues("object-size-unit")[i]; - isRange = false; - } - - // parsing number of objects - String objects[] = req.getParameterValues("num-of-objects")[i] - .split(","); - // parsing number of containers - String containers[] = req.getParameterValues("num-of-containers")[i] - .split(","); - - //Get read, write and delete ratios for one workload in string array. - - String rWDRatios[] = getRWDRatios(req,workloadNumber+""); - - // parsing comma separated worker values - String workers[] = req.getParameterValues("workers")[i].split(","); - - Workload workload = constructWorkload(req, objectSizes, containers, - objects, workers, rWDRatios, - isRange, unit); - - workload.validate(); - - String workloadName = req.getParameterValues("workload.name")[workloadNumber]; - workload.setName(workloadName); - - if (generateWorkloadFiles) - { - WORKLOAD_CONFIG_DIR = new File(workloadConfigFilesRoot+"/"+workloadMatrixName); - if (!WORKLOAD_CONFIG_DIR.exists()) - WORKLOAD_CONFIG_DIR.mkdirs(); - String path = WORKLOAD_CONFIG_DIR.getAbsolutePath(); - LOGGER.info("using {} for storing generated workload configs", path); - - printWorkloadConfigXML(workload, workloadName); - } - else - { - submitWorkload(workload); - } - } - } - - private String[] getRWDRatios(HttpServletRequest req, String workloadNumber) { - String[] readRatios = req.getParameterValues("read-ratio"+workloadNumber); - String[] writeRatios = req.getParameterValues("write-ratio"+workloadNumber); - String[] deleteRatios = req.getParameterValues("delete-ratio"+workloadNumber); - String[] rwdRatios = new String[readRatios.length]; - for(int i=0; i model, HttpServletRequest req, @Override protected ModelAndView process(HttpServletRequest req, HttpServletResponse res) { - - Workload workload = null; - String xml = ""; - + + Workload workload = null; + String xml = ""; + try { - workload = constructWorkloadFromPostData(req); - xml = CastorConfigTools.getWorkloadWriter().toXmlString(workload); + workload = constructWorkloadFromPostData(req); + xml = CastorConfigTools.getWorkloadWriter().toXmlString(workload); } catch (Exception e) { return createErrResult(xml, e.getMessage()); @@ -84,496 +84,496 @@ protected ModelAndView process(HttpServletRequest req, private ArrayList constructInitStage(HttpServletRequest req) { - String initChecked[] = req.getParameterValues("init.checked"); - if (initChecked != null) { - String workStageName = new String("init"); - ArrayList workStageList = new ArrayList(); - for (int i = 0; i < initChecked.length; i++) { - if (i > 0) { - workStageName = new String("init" + i); - } - Stage stage = new Stage(workStageName); - Work work = new Work(workStageName, "init"); - work.setWorkers(getParmInt( - req.getParameterValues("init.workers")[i], 1)); - work.setDivision("container"); - String config = ""; - String selector = req.getParameterValues("init.containers")[i]; - String min = req.getParameterValues("init.containers.min")[i]; - String max = req.getParameterValues("init.containers.max")[i]; - config = "containers=" + selector + "(" + min + "," + max + ")"; - - work.setConfig(config); - - stage.addWork(work); - - workStageList.add(stage); - - checkAndAddDelay(req, "init", workStageList, i); - } - return workStageList; - } - return null; + String initChecked[] = req.getParameterValues("init.checked"); + if (initChecked != null) { + String workStageName = new String("init"); + ArrayList workStageList = new ArrayList(); + for (int i = 0; i < initChecked.length; i++) { + if (i > 0) { + workStageName = new String("init" + i); + } + Stage stage = new Stage(workStageName); + Work work = new Work(workStageName, "init"); + work.setWorkers(getParmInt( + req.getParameterValues("init.workers")[i], 1)); + work.setDivision("container"); + String config = ""; + String selector = req.getParameterValues("init.containers")[i]; + String min = req.getParameterValues("init.containers.min")[i]; + String max = req.getParameterValues("init.containers.max")[i]; + config = "containers=" + selector + "(" + min + "," + max + ")"; + + work.setConfig(config); + + stage.addWork(work); + + workStageList.add(stage); + + checkAndAddDelay(req, "init", workStageList, i); + } + return workStageList; + } + return null; } - - private ArrayList constructPrepareStage(HttpServletRequest req) { - String prepareChecked[] = req.getParameterValues("prepare.checked"); - if (prepareChecked != null) { - String workStageName = new String("prepare"); - ArrayList workStageList = new ArrayList(); - for (int i = 0; i < prepareChecked.length; i++) { - if (i > 0) { - workStageName = new String("prepare" + i); - } - Stage stage = new Stage(workStageName); - Work work = new Work(workStageName, "prepare"); - work.setWorkers(getParmInt( - req.getParameterValues("prepare.workers")[i], 1)); - work.setDivision("object"); - String config = ""; - - // "containers" section in config - String cselector = req.getParameterValues("prepare.containers")[i]; - String cmin = req.getParameterValues("prepare.containers.min")[i]; - String cmax = req.getParameterValues("prepare.containers.max")[i]; - config += "containers=" + cselector + "(" + cmin + "," + cmax - + ");"; - - // "objects" section in config - String oselector = req.getParameterValues("prepare.objects")[i]; - String omin = req.getParameterValues("prepare.objects.min")[i]; - String omax = req.getParameterValues("prepare.objects.max")[i]; - config += "objects=" + oselector + "(" + omin + "," + omax - + ");"; - - // "sizes" section in config - String sselector = getParm(req, "prepare.sizes"); - String smin = req.getParameterValues("prepare.sizes.min")[i]; - String smax = req.getParameterValues("prepare.sizes.max")[i]; - String sunit = req.getParameterValues("prepare.sizes.unit")[i]; - - String sexp = ""; - if ("u".equals(sselector)) - sexp = sselector + "(" + smin + "," + smax + ")" + sunit; - if ("c".equals(sselector)) - sexp = sselector + "(" + smin + ")" + sunit; - - config += "sizes=" + sexp; - - work.setConfig(config); - - stage.addWork(work); - - workStageList.add(stage); - - checkAndAddDelay(req,"prepare",workStageList,i); - } - - return workStageList; - } - - return null; - - } - + + private ArrayList constructPrepareStage(HttpServletRequest req) { + String prepareChecked[] = req.getParameterValues("prepare.checked"); + if (prepareChecked != null) { + String workStageName = new String("prepare"); + ArrayList workStageList = new ArrayList(); + for (int i = 0; i < prepareChecked.length; i++) { + if (i > 0) { + workStageName = new String("prepare" + i); + } + Stage stage = new Stage(workStageName); + Work work = new Work(workStageName, "prepare"); + work.setWorkers(getParmInt( + req.getParameterValues("prepare.workers")[i], 1)); + work.setDivision("object"); + String config = ""; + + // "containers" section in config + String cselector = req.getParameterValues("prepare.containers")[i]; + String cmin = req.getParameterValues("prepare.containers.min")[i]; + String cmax = req.getParameterValues("prepare.containers.max")[i]; + config += "containers=" + cselector + "(" + cmin + "," + cmax + + ");"; + + // "objects" section in config + String oselector = req.getParameterValues("prepare.objects")[i]; + String omin = req.getParameterValues("prepare.objects.min")[i]; + String omax = req.getParameterValues("prepare.objects.max")[i]; + config += "objects=" + oselector + "(" + omin + "," + omax + + ");"; + + // "sizes" section in config + String sselector = getParm(req, "prepare.sizes"); + String smin = req.getParameterValues("prepare.sizes.min")[i]; + String smax = req.getParameterValues("prepare.sizes.max")[i]; + String sunit = req.getParameterValues("prepare.sizes.unit")[i]; + + String sexp = ""; + if ("u".equals(sselector)) + sexp = sselector + "(" + smin + "," + smax + ")" + sunit; + if ("c".equals(sselector)) + sexp = sselector + "(" + smin + ")" + sunit; + + config += "sizes=" + sexp; + + work.setConfig(config); + + stage.addWork(work); + + workStageList.add(stage); + + checkAndAddDelay(req,"prepare",workStageList,i); + } + + return workStageList; + } + + return null; + + } + private ArrayList constructCleanupStage(HttpServletRequest req) { - String cleanupChecked[] = req.getParameterValues("cleanup.checked"); - if (cleanupChecked != null) { - String workStageName = new String("cleanup"); - ArrayList workStageList = new ArrayList(); - for (int i = 0; i < cleanupChecked.length; i++) { - if (i > 0) { - workStageName = new String("cleanup" + i); - } - Stage stage = new Stage(workStageName); - Work work = new Work(workStageName, "cleanup"); - - work.setWorkers(getParmInt( - req.getParameterValues("cleanup.workers")[i], 1)); - work.setDivision("object"); - String config = ""; - - // "containers" section in config - String cselector = req.getParameterValues("cleanup.containers")[i]; - String cmin = req.getParameterValues("cleanup.containers.min")[i]; - String cmax = req.getParameterValues("cleanup.containers.max")[i]; - config += "containers=" + cselector + "(" + cmin + "," + cmax - + ");"; - - // "objects" section in config - String oselector = req.getParameterValues("cleanup.objects")[i]; - String omin = req.getParameterValues("cleanup.objects.min")[i]; - String omax = req.getParameterValues("cleanup.objects.max")[i]; - config += "objects=" + oselector + "(" + omin + "," + omax - + ");"; - - work.setConfig(config); - - stage.addWork(work); - - workStageList.add(stage); - - checkAndAddDelay(req, "cleanup", workStageList, i); - } - - return workStageList; - } - - return null; + String cleanupChecked[] = req.getParameterValues("cleanup.checked"); + if (cleanupChecked != null) { + String workStageName = new String("cleanup"); + ArrayList workStageList = new ArrayList(); + for (int i = 0; i < cleanupChecked.length; i++) { + if (i > 0) { + workStageName = new String("cleanup" + i); + } + Stage stage = new Stage(workStageName); + Work work = new Work(workStageName, "cleanup"); + + work.setWorkers(getParmInt( + req.getParameterValues("cleanup.workers")[i], 1)); + work.setDivision("object"); + String config = ""; + + // "containers" section in config + String cselector = req.getParameterValues("cleanup.containers")[i]; + String cmin = req.getParameterValues("cleanup.containers.min")[i]; + String cmax = req.getParameterValues("cleanup.containers.max")[i]; + config += "containers=" + cselector + "(" + cmin + "," + cmax + + ");"; + + // "objects" section in config + String oselector = req.getParameterValues("cleanup.objects")[i]; + String omin = req.getParameterValues("cleanup.objects.min")[i]; + String omax = req.getParameterValues("cleanup.objects.max")[i]; + config += "objects=" + oselector + "(" + omin + "," + omax + + ");"; + + work.setConfig(config); + + stage.addWork(work); + + workStageList.add(stage); + + checkAndAddDelay(req, "cleanup", workStageList, i); + } + + return workStageList; + } + + return null; } - + private ArrayList constructDisposeStage(HttpServletRequest req) { - String disposeChecked[] = req.getParameterValues("dispose.checked"); - if (disposeChecked != null) { - String workStageName = new String("dispose"); - ArrayList workStageList = new ArrayList(); - for (int i = 0; i < disposeChecked.length; i++) { - if (i > 0) { - workStageName = new String("dispose" + i); - } - Stage stage = new Stage(workStageName); - Work work = new Work(workStageName, "dispose"); - work.setWorkers(getParmInt( - req.getParameterValues("dispose.workers")[i], 1)); - work.setDivision("container"); - String config = ""; - - // "containers" section in config - String cselector = req.getParameterValues("dispose.containers")[i]; - String cmin = req.getParameterValues("dispose.containers.min")[i]; - String cmax = req.getParameterValues("dispose.containers.max")[i]; - config += "containers=" + cselector + "(" + cmin + "," + cmax - + ");"; - - work.setConfig(config); - - stage.addWork(work); - - workStageList.add(stage); - - checkAndAddDelay(req, "dispose", workStageList, i); - } - - return workStageList; - } - - return null; + String disposeChecked[] = req.getParameterValues("dispose.checked"); + if (disposeChecked != null) { + String workStageName = new String("dispose"); + ArrayList workStageList = new ArrayList(); + for (int i = 0; i < disposeChecked.length; i++) { + if (i > 0) { + workStageName = new String("dispose" + i); + } + Stage stage = new Stage(workStageName); + Work work = new Work(workStageName, "dispose"); + work.setWorkers(getParmInt( + req.getParameterValues("dispose.workers")[i], 1)); + work.setDivision("container"); + String config = ""; + + // "containers" section in config + String cselector = req.getParameterValues("dispose.containers")[i]; + String cmin = req.getParameterValues("dispose.containers.min")[i]; + String cmax = req.getParameterValues("dispose.containers.max")[i]; + config += "containers=" + cselector + "(" + cmin + "," + cmax + + ");"; + + work.setConfig(config); + + stage.addWork(work); + + workStageList.add(stage); + + checkAndAddDelay(req, "dispose", workStageList, i); + } + + return workStageList; + } + + return null; } - + private String getParm(HttpServletRequest req, String parm) { - return req.getParameter(parm); + return req.getParameter(parm); } - + // private String getParm(HttpServletRequest req, String parm, String defVal) // { -// String val = getParm(req, parm); -// if(val == null || val.isEmpty()) -// return defVal; -// return val; +// String val = getParm(req, parm); +// if(val == null || val.isEmpty()) +// return defVal; +// return val; // } - + // private int getParmInt(HttpServletRequest req, String parm) // { -// return Integer.parseInt(getParm(req, parm)); +// return Integer.parseInt(getParm(req, parm)); // } - + // private int getParmInt(HttpServletRequest req, String parm, int defVal) // { -// String val = getParm(req, parm); -// if(val == null || val.isEmpty()) -// return defVal; -// -// return Integer.parseInt(val); +// String val = getParm(req, parm); +// if(val == null || val.isEmpty()) +// return defVal; +// +// return Integer.parseInt(val); // } - - private ArrayList constructNormalStage(HttpServletRequest req) { - String normalChecked[] = req.getParameterValues("normal.checked"); - if (normalChecked != null) { - - String workStageName = new String("normal"); - ArrayList workStageList = new ArrayList(); - for (int i = 0; i < normalChecked.length; i++) { - if (i > 0) { - workStageName = new String("normal" + i); - } - Stage stage = new Stage(workStageName); - Work work = new Work(workStageName, "normal"); - work.setWorkers(getParmInt(req - .getParameterValues("normal.workers")[i])); - work.setRampup(getParmInt(req - .getParameterValues("normal.rampup")[i])); - work.setRuntime(getParmInt(req - .getParameterValues("normal.runtime")[i])); - - // read operation - int rRatio = getParmInt( - req.getParameterValues("read.ratio")[i], 0); - - if (rRatio > 0) { - String rconfig = ""; - Operation rOp = new Operation("read"); - rOp.setRatio(rRatio); - - String rcselector = req.getParameterValues("read.containers")[i]; - String rcmin = req.getParameterValues("read.containers.min")[i]; - String rcmax = req.getParameterValues("read.containers.max")[i]; - rconfig += "containers=" + rcselector + "(" + rcmin + "," - + rcmax + ");"; - - // "objects" section in config - String roselector = req.getParameterValues("read.objects")[i]; - String romin = req.getParameterValues("read.objects.min")[i]; - String romax = req.getParameterValues("read.objects.max")[i]; - rconfig += "objects=" + roselector + "(" + romin + "," + romax - + ");"; - rOp.setConfig(rconfig); - - work.addOperation(rOp); - } - - // write operation - int wRatio = getParmInt( - req.getParameterValues("write.ratio")[i], 0); - if (wRatio > 0) { - String wconfig = ""; - Operation wOp = new Operation("write"); - wOp.setRatio(wRatio); - - String wcselector = req.getParameterValues("write.containers")[i]; - String wcmin = req.getParameterValues("write.containers.min")[i]; - String wcmax = req.getParameterValues("write.containers.max")[i]; - wconfig += "containers=" + wcselector + "(" + wcmin + "," - + wcmax + ");"; - - // "objects" section in config - String woselector = req.getParameterValues("write.objects")[i]; - String womin = req.getParameterValues("write.objects.min")[i]; - String womax = req.getParameterValues("write.objects.max")[i]; - wconfig += "objects=" + woselector + "(" + womin + "," + womax - + ");"; - - // "sizes" section in config - String wsselector = req.getParameterValues("write.sizes")[i]; - String wsmin = req.getParameterValues("write.sizes.min")[i]; - String wsmax = req.getParameterValues("write.sizes.max")[i]; - String wsunit = req.getParameterValues("write.sizes.unit")[i]; - - String wsexp = ""; - if ("u".equals(wsselector) || "r".equals(wsselector)) - wsexp = wsselector + "(" + wsmin + "," + wsmax + ")" - + wsunit; - if ("c".equals(wsselector)) - wsexp = wsselector + "(" + wsmin + ")" + wsunit; - - wconfig += "sizes=" + wsexp; - - wOp.setConfig(wconfig); - - work.addOperation(wOp); - } - - // filewrite operation - int fwRatio = getParmInt( - req.getParameterValues("filewrite.ratio")[i], 0); - if (fwRatio > 0) { - String fwconfig = ""; - Operation fwOp = new Operation("filewrite"); - fwOp.setRatio(fwRatio); - - // "containers" section in config - String fwcselector = req - .getParameterValues("filewrite.containers")[i]; - String fwcmin = req - .getParameterValues("filewrite.containers.min")[i]; - String fwcmax = req - .getParameterValues("filewrite.containers.max")[i]; - fwconfig += "containers=" + fwcselector + "(" + fwcmin + "," - + fwcmax + ");"; - - // "objects" section in config - String fwoselector = req - .getParameterValues("filewrite.fileselection")[i]; - fwconfig += "fileselection=" + fwoselector + ";"; - - // "files" section in config - String fwfselector = req.getParameterValues("filewrite.files")[i]; - fwconfig += "files=" + fwfselector; - - fwOp.setConfig(fwconfig); - - work.addOperation(fwOp); - } - - // delete operation - int dRatio = getParmInt( - req.getParameterValues("delete.ratio")[i], 0); - if (dRatio > 0) { - String dconfig = ""; - Operation dOp = new Operation("delete"); - - - dOp.setRatio(dRatio); - - String dcselector = req.getParameterValues("delete.containers")[i]; - String dcmin = req.getParameterValues("delete.containers.min")[i]; - String dcmax = req.getParameterValues("delete.containers.max")[i]; - dconfig += "containers=" + dcselector + "(" + dcmin + "," - + dcmax + ");"; - - // "objects" section in config - String doselector = req.getParameterValues("delete.objects")[i]; - String domin = req.getParameterValues("delete.objects.min")[i]; - String domax = req.getParameterValues("delete.objects.max")[i]; - dconfig += "objects=" + doselector + "(" + domin + "," + domax - + ");"; - dOp.setConfig(dconfig); - - work.addOperation(dOp); - } - - stage.addWork(work); - - workStageList.add(stage); - - checkAndAddDelay(req, "normal", workStageList, i); - } - return workStageList; - } - - return null; - } - - private int getParmInt(String string, int defVal) { - if (string != null) - return Integer.parseInt(string); - else - return defVal; - } - - private int getParmInt(String string) { - return Integer.parseInt(string); - } - - private void checkAndAddDelay(HttpServletRequest req, String stage, - ArrayList workStageList, int iteration) { - String delayChecked[] = req - .getParameterValues(stage + ".delay.checked"); - if (delayChecked != null) { - boolean hasDelay = ("on".equalsIgnoreCase(delayChecked[iteration])); - if (hasDelay) { - Stage delayStage = new Stage("delay"); - delayStage - .setClosuredelay(getParmInt( - req.getParameterValues("init.delay.closuredelay")[iteration], - 60)); - Work work = new Work("delay", "delay"); - work.addOperation(new Operation("delay")); - delayStage.addWork(work); - workStageList.add(delayStage); - } - } - } - - // method for removing nsroot config from prepare, normal and cleanup stages - private Storage removeNSROOTConfig(Storage storage) { - if(storage.getConfig() == null) - return storage; - if (!storage.getConfig().contains("nsroot")) - return storage; - else { - Storage newStorage = new Storage(); - String configParams[] = storage.getConfig().split(";"); - StringBuffer newConfig = new StringBuffer(""); - for (String configParam : configParams) { - if (!configParam.toLowerCase().contains("nsroot")) - newConfig.append(configParam + ";"); - } - newConfig.deleteCharAt(newConfig.length() - 1); - newStorage.setType(storage.getType()); - newStorage.setConfig(newConfig.toString()); - return newStorage; - } - } + + private ArrayList constructNormalStage(HttpServletRequest req) { + String normalChecked[] = req.getParameterValues("normal.checked"); + if (normalChecked != null) { + + String workStageName = new String("normal"); + ArrayList workStageList = new ArrayList(); + for (int i = 0; i < normalChecked.length; i++) { + if (i > 0) { + workStageName = new String("normal" + i); + } + Stage stage = new Stage(workStageName); + Work work = new Work(workStageName, "normal"); + work.setWorkers(getParmInt(req + .getParameterValues("normal.workers")[i])); + work.setRampup(getParmInt(req + .getParameterValues("normal.rampup")[i])); + work.setRuntime(getParmInt(req + .getParameterValues("normal.runtime")[i])); + + // read operation + int rRatio = getParmInt( + req.getParameterValues("read.ratio")[i], 0); + + if (rRatio > 0) { + String rconfig = ""; + Operation rOp = new Operation("read"); + rOp.setRatio(rRatio); + + String rcselector = req.getParameterValues("read.containers")[i]; + String rcmin = req.getParameterValues("read.containers.min")[i]; + String rcmax = req.getParameterValues("read.containers.max")[i]; + rconfig += "containers=" + rcselector + "(" + rcmin + "," + + rcmax + ");"; + + // "objects" section in config + String roselector = req.getParameterValues("read.objects")[i]; + String romin = req.getParameterValues("read.objects.min")[i]; + String romax = req.getParameterValues("read.objects.max")[i]; + rconfig += "objects=" + roselector + "(" + romin + "," + romax + + ");"; + rOp.setConfig(rconfig); + + work.addOperation(rOp); + } + + // write operation + int wRatio = getParmInt( + req.getParameterValues("write.ratio")[i], 0); + if (wRatio > 0) { + String wconfig = ""; + Operation wOp = new Operation("write"); + wOp.setRatio(wRatio); + + String wcselector = req.getParameterValues("write.containers")[i]; + String wcmin = req.getParameterValues("write.containers.min")[i]; + String wcmax = req.getParameterValues("write.containers.max")[i]; + wconfig += "containers=" + wcselector + "(" + wcmin + "," + + wcmax + ");"; + + // "objects" section in config + String woselector = req.getParameterValues("write.objects")[i]; + String womin = req.getParameterValues("write.objects.min")[i]; + String womax = req.getParameterValues("write.objects.max")[i]; + wconfig += "objects=" + woselector + "(" + womin + "," + womax + + ");"; + + // "sizes" section in config + String wsselector = req.getParameterValues("write.sizes")[i]; + String wsmin = req.getParameterValues("write.sizes.min")[i]; + String wsmax = req.getParameterValues("write.sizes.max")[i]; + String wsunit = req.getParameterValues("write.sizes.unit")[i]; + + String wsexp = ""; + if ("u".equals(wsselector) || "r".equals(wsselector)) + wsexp = wsselector + "(" + wsmin + "," + wsmax + ")" + + wsunit; + if ("c".equals(wsselector)) + wsexp = wsselector + "(" + wsmin + ")" + wsunit; + + wconfig += "sizes=" + wsexp; + + wOp.setConfig(wconfig); + + work.addOperation(wOp); + } + + // filewrite operation + int fwRatio = getParmInt( + req.getParameterValues("filewrite.ratio")[i], 0); + if (fwRatio > 0) { + String fwconfig = ""; + Operation fwOp = new Operation("filewrite"); + fwOp.setRatio(fwRatio); + + // "containers" section in config + String fwcselector = req + .getParameterValues("filewrite.containers")[i]; + String fwcmin = req + .getParameterValues("filewrite.containers.min")[i]; + String fwcmax = req + .getParameterValues("filewrite.containers.max")[i]; + fwconfig += "containers=" + fwcselector + "(" + fwcmin + "," + + fwcmax + ");"; + + // "objects" section in config + String fwoselector = req + .getParameterValues("filewrite.fileselection")[i]; + fwconfig += "fileselection=" + fwoselector + ";"; + + // "files" section in config + String fwfselector = req.getParameterValues("filewrite.files")[i]; + fwconfig += "files=" + fwfselector; + + fwOp.setConfig(fwconfig); + + work.addOperation(fwOp); + } + + // delete operation + int dRatio = getParmInt( + req.getParameterValues("delete.ratio")[i], 0); + if (dRatio > 0) { + String dconfig = ""; + Operation dOp = new Operation("delete"); + + + dOp.setRatio(dRatio); + + String dcselector = req.getParameterValues("delete.containers")[i]; + String dcmin = req.getParameterValues("delete.containers.min")[i]; + String dcmax = req.getParameterValues("delete.containers.max")[i]; + dconfig += "containers=" + dcselector + "(" + dcmin + "," + + dcmax + ");"; + + // "objects" section in config + String doselector = req.getParameterValues("delete.objects")[i]; + String domin = req.getParameterValues("delete.objects.min")[i]; + String domax = req.getParameterValues("delete.objects.max")[i]; + dconfig += "objects=" + doselector + "(" + domin + "," + domax + + ");"; + dOp.setConfig(dconfig); + + work.addOperation(dOp); + } + + stage.addWork(work); + + workStageList.add(stage); + + checkAndAddDelay(req, "normal", workStageList, i); + } + return workStageList; + } + + return null; + } + + private int getParmInt(String string, int defVal) { + if (string != null) + return Integer.parseInt(string); + else + return defVal; + } + + private int getParmInt(String string) { + return Integer.parseInt(string); + } + + private void checkAndAddDelay(HttpServletRequest req, String stage, + ArrayList workStageList, int iteration) { + String delayChecked[] = req + .getParameterValues(stage + ".delay.checked"); + if (delayChecked != null) { + boolean hasDelay = ("on".equalsIgnoreCase(delayChecked[iteration])); + if (hasDelay) { + Stage delayStage = new Stage("delay"); + delayStage + .setClosuredelay(getParmInt( + req.getParameterValues("init.delay.closuredelay")[iteration], + 60)); + Work work = new Work("delay", "delay"); + work.addOperation(new Operation("delay")); + delayStage.addWork(work); + workStageList.add(delayStage); + } + } + } + + // method for removing nsroot config from prepare, normal and cleanup stages + private Storage removeNSROOTConfig(Storage storage) { + if(storage.getConfig() == null) + return storage; + if (!storage.getConfig().contains("nsroot")) + return storage; + else { + Storage newStorage = new Storage(); + String configParams[] = storage.getConfig().split(";"); + StringBuffer newConfig = new StringBuffer(""); + for (String configParam : configParams) { + if (!configParam.toLowerCase().contains("nsroot")) + newConfig.append(configParam + ";"); + } + newConfig.deleteCharAt(newConfig.length() - 1); + newStorage.setType(storage.getType()); + newStorage.setConfig(newConfig.toString()); + return newStorage; + } + } private Workload constructWorkloadFromPostData(HttpServletRequest req) throws Exception { - Workload workload = new Workload(); - - String name = getParm(req, "workload.name"); - if(name == null || name.isEmpty()) - name = "workload"; - String desc = getParm(req, "workload.desc"); - - workload.setName(name); - workload.setDescription(desc); - - workload.setAuth(new Auth(getParm(req, "auth.type"), getParm(req, "auth.config"))); - workload.setStorage(new Storage(getParm(req, "storage.type"), getParm(req, "storage.config"))); - - Workflow workflow = new Workflow(); - - ArrayList initStageList = constructInitStage(req); - if (initStageList != null) { - for (int i = 0; i < initStageList.size(); i++) { - workflow.addStage((Stage) initStageList.get(i)); - } - } - - ArrayList prepareStageList = constructPrepareStage(req); - if (prepareStageList != null) { - for (int i = 0; i < prepareStageList.size(); i++) { - Stage stage = (Stage) prepareStageList.get(i); - stage.setStorage(removeNSROOTConfig(workload.getStorage())); - workflow.addStage(stage); - } - } - - ArrayList normalStageList = constructNormalStage(req); - if (normalStageList != null) { - for (int i = 0; i < normalStageList.size(); i++) { - Stage stage = (Stage) normalStageList.get(i); - stage.setStorage(removeNSROOTConfig(workload.getStorage())); - workflow.addStage(stage); - } - } - - ArrayList cleanupStageList = constructCleanupStage(req); - if (cleanupStageList != null) { - for (int i = 0; i < cleanupStageList.size(); i++) { - Stage stage = (Stage) cleanupStageList.get(i); - stage.setStorage(removeNSROOTConfig(workload.getStorage())); - workflow.addStage(stage); - } - } - - ArrayList disposeStageList = constructDisposeStage(req); - if (disposeStageList != null) { - for (int i = 0; i < disposeStageList.size(); i++) { - workflow.addStage((Stage) disposeStageList.get(i)); - } - } - - workload.setWorkflow(workflow); - - workload.validate(); - - return workload; + Workload workload = new Workload(); + + String name = getParm(req, "workload.name"); + if(name == null || name.isEmpty()) + name = "workload"; + String desc = getParm(req, "workload.desc"); + + workload.setName(name); + workload.setDescription(desc); + + workload.setAuth(new Auth(getParm(req, "auth.type"), getParm(req, "auth.config"))); + workload.setStorage(new Storage(getParm(req, "storage.type"), getParm(req, "storage.config"))); + + Workflow workflow = new Workflow(); + + ArrayList initStageList = constructInitStage(req); + if (initStageList != null) { + for (int i = 0; i < initStageList.size(); i++) { + workflow.addStage((Stage) initStageList.get(i)); + } + } + + ArrayList prepareStageList = constructPrepareStage(req); + if (prepareStageList != null) { + for (int i = 0; i < prepareStageList.size(); i++) { + Stage stage = (Stage) prepareStageList.get(i); + stage.setStorage(removeNSROOTConfig(workload.getStorage())); + workflow.addStage(stage); + } + } + + ArrayList normalStageList = constructNormalStage(req); + if (normalStageList != null) { + for (int i = 0; i < normalStageList.size(); i++) { + Stage stage = (Stage) normalStageList.get(i); + stage.setStorage(removeNSROOTConfig(workload.getStorage())); + workflow.addStage(stage); + } + } + + ArrayList cleanupStageList = constructCleanupStage(req); + if (cleanupStageList != null) { + for (int i = 0; i < cleanupStageList.size(); i++) { + Stage stage = (Stage) cleanupStageList.get(i); + stage.setStorage(removeNSROOTConfig(workload.getStorage())); + workflow.addStage(stage); + } + } + + ArrayList disposeStageList = constructDisposeStage(req); + if (disposeStageList != null) { + for (int i = 0; i < disposeStageList.size(); i++) { + workflow.addStage((Stage) disposeStageList.get(i)); + } + } + + workload.setWorkflow(workflow); + + workload.validate(); + + return workload; } - + private ModelAndView createErrResult(String xml, String msg) { ModelAndView result = new ModelAndView("config", "xml", xml); result.addObject("error", "ERROR: " + msg); - + result.addObject("cInfo", controller.getControllerInfo()); return result; } - private ModelAndView createSuccResult(String xml) { + private ModelAndView createSuccResult(String xml) { ModelAndView result = new ModelAndView(XML, "xml", xml); - + result.addObject("cInfo", controller.getControllerInfo()); return result; } - + } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadMatrixConfigurationController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadMatrixConfigurationController.java index 2691072e..68e73a31 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadMatrixConfigurationController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadMatrixConfigurationController.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.web; @@ -12,20 +30,20 @@ import com.intel.cosbench.service.ControllerService; import com.intel.cosbench.web.AbstractController; -public class WorkloadMatrixConfigurationController extends AbstractController { +public class WorkloadMatrixConfigurationController extends AbstractController { protected ControllerService controller; public void setController(ControllerService controller) { this.controller = controller; } - + @Override protected ModelAndView process(HttpServletRequest req, HttpServletResponse res) { - + try { - constructWorkloadConfigsFromPostData(req); + constructWorkloadConfigsFromPostData(req); } catch (Exception e) { return createErrResult(e.getMessage()); @@ -34,23 +52,26 @@ protected ModelAndView process(HttpServletRequest req, return createSuccResult(); } - private void constructWorkloadConfigsFromPostData(HttpServletRequest req) throws Exception { - - WorkloadConfigGenerator wlConfGen = new WorkloadConfigGenerator(controller); - wlConfGen.createWorkloadFiles(req); - - } - - private ModelAndView createErrResult(String msg) { - ModelAndView result = new ModelAndView("advanced-config"); - result.addObject("error", "ERROR: " + msg); - return result; - } - - private ModelAndView createSuccResult() { - WorkloadInfo[] aInfos = controller.getActiveWorkloads(); - return new ModelAndView("submit", "aInfos", aInfos); - } - + private void constructWorkloadConfigsFromPostData(HttpServletRequest req) throws Exception { + + WorkloadConfigGenerator wlConfGen = new WorkloadConfigGenerator(controller); + wlConfGen.createWorkloadFiles(req); + + } + + private ModelAndView createErrResult(String msg) { + ModelAndView result = new ModelAndView("advanced-config"); + result.addObject("error", "ERROR: " + msg); + result.addObject("cInfo", controller.getControllerInfo()); + return result; + } + + private ModelAndView createSuccResult() { + WorkloadInfo[] aInfos = controller.getActiveWorkloads(); + ModelAndView result = new ModelAndView("submit", "aInfos", aInfos); + result.addObject("cInfo", controller.getControllerInfo()); + return result; + } + } diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadPageController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadPageController.java index 385ef987..21bfc915 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadPageController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -51,19 +51,20 @@ protected ModelAndView process(String id) { WorkloadInfo info = controller.getWorkloadInfo(id); if (info == null) throw new NotFoundException(); - if (controller.getloadArch() && info.getArchived() && info.getReport().getAllMetrics().length==0) { - try { - controller.getWorkloadLoader().loadWorkloadPageInfo(info); - } catch (IOException e) { - e.printStackTrace(); - } - } + if (controller.getloadArch() && info.getArchived() && info.getReport().getAllMetrics().length==0) { + try { + controller.getWorkloadLoader().loadWorkloadPageInfo(info); + } catch (IOException e) { + e.printStackTrace(); + } + } return process(info); } protected ModelAndView process(WorkloadInfo info) { ModelAndView result = new ModelAndView("workload"); StageInfo stage = info.getCurrentStage(); + result.addObject("cInfo", controller.getControllerInfo()); result.addObject("info", info); result.addObject("isStopped", isStopped(info.getState())); result.addObject("isRunning", isRunning(info.getState())); diff --git a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadSubmissionController.java b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadSubmissionController.java index 403a626b..f4a7856b 100644 --- a/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadSubmissionController.java +++ b/dev/cosbench-controller-web/src/com/intel/cosbench/controller/web/WorkloadSubmissionController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.web; @@ -68,6 +68,7 @@ private ModelAndView createErrResult(String msg) { ModelAndView result = new ModelAndView("submit"); result.addObject("aInfos", controller.getActiveWorkloads()); result.addObject("error", msg); + result.addObject("cInfo", controller.getControllerInfo()); return result; } @@ -76,6 +77,7 @@ private ModelAndView createSuccResult(String id) { result.addObject("aInfos", controller.getActiveWorkloads()); result.addObject("submitted", "your workload has been accepted"); result.addObject("id", id); + result.addObject("cInfo", controller.getControllerInfo()); return result; } diff --git a/dev/cosbench-controller/META-INF/MANIFEST.MF b/dev/cosbench-controller/META-INF/MANIFEST.MF index ace9fa08..d7f4fb11 100644 --- a/dev/cosbench-controller/META-INF/MANIFEST.MF +++ b/dev/cosbench-controller/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Controller Bundler Bundle-SymbolicName: cosbench-controller -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.bench, diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/archiver/SimpleWorkloadArchiver.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/archiver/SimpleWorkloadArchiver.java index 13f00367..6c1e466d 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/archiver/SimpleWorkloadArchiver.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/archiver/SimpleWorkloadArchiver.java @@ -1,316 +1,356 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.controller.archiver; - -import java.io.*; -import java.util.Scanner; - -import com.intel.cosbench.config.*; -import com.intel.cosbench.config.castor.CastorConfigTools; -import com.intel.cosbench.exporter.*; -import com.intel.cosbench.log.*; -import com.intel.cosbench.model.*; - -/** - * This class encapsulates operations to archive workload results, the archive - * folder is at "$CURRENT/archive". - * - * @author ywang19, qzheng7 - * - */ -public class SimpleWorkloadArchiver implements WorkloadArchiver { - - private static final Logger LOGGER = LogFactory.getSystemLogger(); - - private File ARCHIVE_DIR = new File("archive"); - -// private static final File ROOT_DIR = new File("archive"); - -// static { -// if (!ROOT_DIR.exists()) -// ROOT_DIR.mkdirs(); -// String path = ROOT_DIR.getAbsolutePath(); -// LOGGER.info("using {} for storing workload archives", path); -// } - - public SimpleWorkloadArchiver() { - this("archive"); - } - - public SimpleWorkloadArchiver(final String archive) { - ARCHIVE_DIR = new File(archive); - - if (!ARCHIVE_DIR.exists()) - ARCHIVE_DIR.mkdirs(); - String path = ARCHIVE_DIR.getAbsolutePath(); - LOGGER.info("using {} for storing workload archives", path); - } - - @Override - public synchronized void archive(WorkloadInfo info) { - File runDir = new File(ARCHIVE_DIR, getRunDirName(info)); - try { - doArchive(info, runDir); - } catch (Exception e) { - LOGGER.error("fail to archive workload", e); - return; - } - try { - updateCount(info); - } catch (Exception e) { - LOGGER.error("fail to update count", e); - } - String id = info.getId(); - LOGGER.info("workload {} has been successfully archived", id); - } - - private void doArchive(WorkloadInfo info, File runDir) throws IOException { - exportWorkloadRun(info); - runDir.mkdir(); - exportWorkload(info, runDir); - exportLatency(info, runDir); - for (StageInfo sInfo : info.getStageInfos()) - exportStage(sInfo, runDir); - exportConfig(info.getWorkload(), runDir); - exportLog(info, runDir); - exportScriptsLog(info, runDir); - exportPerformanceMatrix(info); - } - - private static String getRunDirName(WorkloadInfo info) { - String name = info.getId(); - name += '-' + info.getWorkload().getName(); - return name; - } - - private void exportWorkloadRun(WorkloadInfo info) throws IOException { - File file = new File(ARCHIVE_DIR, "run-history.csv"); - boolean ready = file.exists() && file.length() > 0; - Writer writer = new BufferedWriter(new FileWriter(file, true)); - RunExporter exporter = Exporters.newRunExporter(info); - try { - if (!ready) - exporter.init(writer); - exporter.export(writer); - } finally { - writer.close(); - } - String id = info.getId(); - String path = file.getAbsolutePath(); - String msg = "run item for workload {} has been added to {}"; - LOGGER.debug(msg, id, path); - } - - private void exportWorkload(WorkloadInfo info, File parent) - throws IOException { - File file = new File(parent, getWorkloadFileName(info) + ".csv"); - Writer writer = new BufferedWriter(new FileWriter(file)); - WorkloadExporter exporter = Exporters.newWorkloadExporter(info); - try { - exporter.export(writer); - } finally { - writer.close(); - } - String id = info.getId(); - String path = file.getAbsolutePath(); - String msg = "perf details of workload {} has been exported to {}"; - LOGGER.debug(msg, id, path); - } - - private static String getWorkloadFileName(WorkloadInfo info) { - String name = info.getId(); - name += "-" + info.getWorkload().getName(); - return name; - } - - private void exportLatency(WorkloadInfo info, File parent) - throws IOException { - File file = new File(parent, getLatencyFileName(info) + ".csv"); - Writer writer = new BufferedWriter(new FileWriter(file)); - LatencyExporter exporter = Exporters.newLatencyExporter(info); - try { - exporter.export(writer); - } finally { - writer.close(); - } - String id = info.getId(); - String path = file.getAbsolutePath(); - String msg = "latency details of workload {} has been exported to {}"; - LOGGER.debug(msg, id, path); - } - - private static String getLatencyFileName(WorkloadInfo info) { - String name = info.getId(); - name += "-" + info.getWorkload().getName(); - name += "-rt-histogram"; - return name; - } - - private void exportStage(StageInfo info, File parent) throws IOException { - File file = new File(parent, getStageFileName(info) + ".csv"); - Writer writer = new BufferedWriter(new FileWriter(file)); - StageExporter exporter = Exporters.newStageExporter(info); - try { - exporter.export(writer); - } finally { - writer.close(); - } - String id = info.getId(); - String path = file.getAbsolutePath(); - String msg = "perf details of stage {} has been exported to {}"; - LOGGER.debug(msg, id, path); - } - - private static String getStageFileName(StageInfo info) { - return info.getId(); - } - - private void exportConfig(Workload workload, File parent) - throws IOException { - File file = new File(parent, "workload-config.xml"); - Writer writer = new BufferedWriter(new FileWriter(file)); - WorkloadWriter ww = CastorConfigTools.getWorkloadWriter(); - try { - writer.write(ww.toXmlString(workload)); - } finally { - writer.close(); - } - String name = workload.getName(); - String path = file.getAbsolutePath(); - String msg = "config xml for workload {} has been generated at {}"; - LOGGER.debug(msg, name, path); - } - - @Override - public File getWorkloadConfig(WorkloadInfo info) { - File runDir = new File(ARCHIVE_DIR, getRunDirName(info)); - return new File(runDir, "workload-config.xml"); - } - - private void exportLog(WorkloadInfo info, File parent) throws IOException { - File file = new File(parent, "workload.log"); - Writer writer = new BufferedWriter(new FileWriter(file)); - LogExporter exporter = Exporters.newLogExporter(info); - try { - exporter.export(writer); - } finally { - writer.close(); - } - String id = info.getId(); - String path = file.getAbsolutePath(); - String msg = "driver logs for workload {} has been merged at {}"; - LOGGER.debug(msg, id, path); - } - - - private void exportScriptsLog(WorkloadInfo info, File parent) throws IOException { - File file = new File(parent, "scripts.log"); - Writer writer = new BufferedWriter(new FileWriter(file)); - LogExporter exporter = Exporters.newScriptLogExporter(info); - try { - exporter.export(writer); - } finally { - writer.close(); - } - String id = info.getId(); - String path = file.getAbsolutePath(); - String msg = "driver script logs for workload {} has been merged at {}"; - LOGGER.debug(msg, id, path); - } - - @Override - public File getWorkloadLog(WorkloadInfo info) { - File runDir = new File(ARCHIVE_DIR, getRunDirName(info)); - return new File(runDir, "workload.log"); - } - - private void exportPerformanceMatrix(WorkloadInfo info) throws IOException { - File file = new File(ARCHIVE_DIR, "workloads.csv"); - boolean ready = file.exists() && file.length() > 0; - Writer writer = new BufferedWriter(new FileWriter(file, true)); - MatrixExporter exporter = Exporters.newMatrixExporter(info); - try { - if (!ready) - exporter.init(writer); - exporter.export(writer); - } finally { - writer.close(); - } - String id = info.getId(); - String path = file.getAbsolutePath(); - String msg = "perf details of workload {} has been added to {}"; - LOGGER.debug(msg, id, path); - } - - private void updateCount(WorkloadInfo info) throws IOException { - int count = 0; - File file = new File(ARCHIVE_DIR, ".meta"); - String workloadId = null; - workloadId = info.getId(); - try { - count = parseID(workloadId); - } catch (Exception e) { - LOGGER.error("cannot parse workloadId", e); - } - - Writer writer = new BufferedWriter(new FileWriter(file)); - try { - writer.write(String.valueOf(count)); - } finally { - writer.close(); - } - LOGGER.debug("workload count has been updated as {}", count); - } - - @Override - public synchronized int getTotalWorkloads() { - int count = 0; - try { - count = retrieveCount(); - } catch (Exception e) { - LOGGER.error("fail to retrieve workload count", e); - } - return count; - } - - private int retrieveCount() throws IOException { - File file = new File(ARCHIVE_DIR, ".meta"); - if (!file.exists() || file.length() == 0) - return 0; - int count; - Reader reader = new BufferedReader(new FileReader(file)); - try { - count = new Scanner(reader).nextInt(); - } finally { - reader.close(); - } - LOGGER.debug("workload count has been retrieved as {}", count); - return count; - } - - private int parseID(String workloadID) throws NumberFormatException, IndexOutOfBoundsException { - int count = 0; - try { - count = Integer.parseInt(workloadID.substring(1)); - } catch (Exception e) { - LOGGER.error("cannot extract count from workloadId", e); - } - return count; - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.controller.archiver; + +import java.io.*; +import java.util.List; +import java.util.Scanner; + +import com.intel.cosbench.bench.Metrics; +import com.intel.cosbench.bench.TaskReport; +import com.intel.cosbench.config.*; +import com.intel.cosbench.config.castor.CastorConfigTools; +import com.intel.cosbench.exporter.*; +import com.intel.cosbench.log.*; +import com.intel.cosbench.model.*; + +/** + * This class encapsulates operations to archive workload results, the archive + * folder is at "$CURRENT/archive". + * + * @author ywang19, qzheng7 + * + */ +public class SimpleWorkloadArchiver implements WorkloadArchiver { + + private static final Logger LOGGER = LogFactory.getSystemLogger(); + + private File ARCHIVE_DIR = new File("archive"); + +// private static final File ROOT_DIR = new File("archive"); + +// static { +// if (!ROOT_DIR.exists()) +// ROOT_DIR.mkdirs(); +// String path = ROOT_DIR.getAbsolutePath(); +// LOGGER.info("using {} for storing workload archives", path); +// } + + public SimpleWorkloadArchiver() { + this("archive"); + } + + public SimpleWorkloadArchiver(final String archive) { + ARCHIVE_DIR = new File(archive); + + if (!ARCHIVE_DIR.exists()) + ARCHIVE_DIR.mkdirs(); + String path = ARCHIVE_DIR.getAbsolutePath(); + LOGGER.info("using {} for storing workload archives", path); + } + + @Override + public synchronized void archive(WorkloadInfo info) { + File runDir = new File(ARCHIVE_DIR, getRunDirName(info)); + try { + doArchive(info, runDir); + } catch (Exception e) { + LOGGER.error("fail to archive workload", e); + return; + } + try { + updateCount(info); + } catch (Exception e) { + LOGGER.error("fail to update count", e); + } + String id = info.getId(); + LOGGER.info("workload {} has been successfully archived", id); + } + + private void doArchive(WorkloadInfo info, File runDir) throws IOException { + exportWorkloadRun(info); + runDir.mkdir(); + exportWorkload(info, runDir); + exportLatency(info, runDir); + for (StageInfo sInfo : info.getStageInfos()) + exportStage(sInfo, runDir); + exportConfig(info.getWorkload(), runDir); + exportLog(info, runDir); + exportScriptsLog(info, runDir); + exportPerformanceMatrix(info); + exportTaskInfo(info,runDir); + exportWorkerInfo(info,runDir); + } + + private void exportWorkerInfo(WorkloadInfo info,File parent)throws IOException{ + for(StageInfo sInfo :info.getStageInfos()){ + File file = new File(parent,getStageFileName(sInfo) +"-worker"+ ".csv"); + Writer writer = new BufferedWriter(new FileWriter(file)); + WorkerExporter exporter = Exporters.newWorkExporter(sInfo); + try { + exporter.export(writer); + } finally { + writer.close(); + } + String name = sInfo.getId()+"worker"; + String path = file.getAbsolutePath(); + String msg = "perf details of {} has been exported to {}"; + LOGGER.debug(msg, name, path); + } + } + + private void exportTaskInfo(WorkloadInfo info,File parent)throws IOException{ + for(DriverInfo dInfo:info.getDriverInfos()){ + File file = new File(parent, dInfo.getName() + ".csv"); + Writer writer = new BufferedWriter(new FileWriter(file)); + TaskExporter exporter = Exporters.newTaskExporter(info,dInfo); + try { + exporter.export(writer); + } finally { + writer.close(); + } + String name = dInfo.getName(); + String path = file.getAbsolutePath(); + String msg = "perf details of {} has been exported to {}"; + LOGGER.debug(msg, name, path); + } + } + + + private static String getRunDirName(WorkloadInfo info) { + String name = info.getId(); + name += '-' + info.getWorkload().getName(); + return name; + } + + private void exportWorkloadRun(WorkloadInfo info) throws IOException { + File file = new File(ARCHIVE_DIR, "run-history.csv"); + boolean ready = file.exists() && file.length() > 0; + Writer writer = new BufferedWriter(new FileWriter(file, true)); + RunExporter exporter = Exporters.newRunExporter(info); + try { + if (!ready) + exporter.init(writer); + exporter.export(writer); + } finally { + writer.close(); + } + String id = info.getId(); + String path = file.getAbsolutePath(); + String msg = "run item for workload {} has been added to {}"; + LOGGER.debug(msg, id, path); + } + + private void exportWorkload(WorkloadInfo info, File parent) + throws IOException { + File file = new File(parent, getWorkloadFileName(info) + ".csv"); + Writer writer = new BufferedWriter(new FileWriter(file)); + WorkloadExporter exporter = Exporters.newWorkloadExporter(info); + try { + exporter.export(writer); + } finally { + writer.close(); + } + String id = info.getId(); + String path = file.getAbsolutePath(); + String msg = "perf details of workload {} has been exported to {}"; + LOGGER.debug(msg, id, path); + } + + private static String getWorkloadFileName(WorkloadInfo info) { + String name = info.getId(); + name += "-" + info.getWorkload().getName(); + return name; + } + + private void exportLatency(WorkloadInfo info, File parent) + throws IOException { + File file = new File(parent, getLatencyFileName(info) + ".csv"); + Writer writer = new BufferedWriter(new FileWriter(file)); + LatencyExporter exporter = Exporters.newLatencyExporter(info); + try { + exporter.export(writer); + } finally { + writer.close(); + } + String id = info.getId(); + String path = file.getAbsolutePath(); + String msg = "latency details of workload {} has been exported to {}"; + LOGGER.debug(msg, id, path); + } + + private static String getLatencyFileName(WorkloadInfo info) { + String name = info.getId(); + name += "-" + info.getWorkload().getName(); + name += "-rt-histogram"; + return name; + } + + private void exportStage(StageInfo info, File parent) throws IOException { + File file = new File(parent, getStageFileName(info) + ".csv"); + Writer writer = new BufferedWriter(new FileWriter(file)); + StageExporter exporter = Exporters.newStageExporter(info); + try { + exporter.export(writer); + } finally { + writer.close(); + } + String id = info.getId(); + String path = file.getAbsolutePath(); + String msg = "perf details of stage {} has been exported to {}"; + LOGGER.debug(msg, id, path); + } + + private static String getStageFileName(StageInfo info) { + return info.getId(); + } + + private void exportConfig(Workload workload, File parent) + throws IOException { + File file = new File(parent, "workload-config.xml"); + Writer writer = new BufferedWriter(new FileWriter(file)); + WorkloadWriter ww = CastorConfigTools.getWorkloadWriter(); + try { + writer.write(ww.toXmlString(workload)); + } finally { + writer.close(); + } + String name = workload.getName(); + String path = file.getAbsolutePath(); + String msg = "config xml for workload {} has been generated at {}"; + LOGGER.debug(msg, name, path); + } + + @Override + public File getWorkloadConfig(WorkloadInfo info) { + File runDir = new File(ARCHIVE_DIR, getRunDirName(info)); + return new File(runDir, "workload-config.xml"); + } + + private void exportLog(WorkloadInfo info, File parent) throws IOException { + File file = new File(parent, "workload.log"); + Writer writer = new BufferedWriter(new FileWriter(file)); + LogExporter exporter = Exporters.newLogExporter(info); + try { + exporter.export(writer); + } finally { + writer.close(); + } + String id = info.getId(); + String path = file.getAbsolutePath(); + String msg = "driver logs for workload {} has been merged at {}"; + LOGGER.debug(msg, id, path); + } + + + private void exportScriptsLog(WorkloadInfo info, File parent) throws IOException { + File file = new File(parent, "scripts.log"); + Writer writer = new BufferedWriter(new FileWriter(file)); + LogExporter exporter = Exporters.newScriptLogExporter(info); + try { + exporter.export(writer); + } finally { + writer.close(); + } + String id = info.getId(); + String path = file.getAbsolutePath(); + String msg = "driver script logs for workload {} has been merged at {}"; + LOGGER.debug(msg, id, path); + } + + @Override + public File getWorkloadLog(WorkloadInfo info) { + File runDir = new File(ARCHIVE_DIR, getRunDirName(info)); + return new File(runDir, "workload.log"); + } + + private void exportPerformanceMatrix(WorkloadInfo info) throws IOException { + File file = new File(ARCHIVE_DIR, "workloads.csv"); + boolean ready = file.exists() && file.length() > 0; + Writer writer = new BufferedWriter(new FileWriter(file, true)); + MatrixExporter exporter = Exporters.newMatrixExporter(info); + try { + if (!ready) + exporter.init(writer); + exporter.export(writer); + } finally { + writer.close(); + } + String id = info.getId(); + String path = file.getAbsolutePath(); + String msg = "perf details of workload {} has been added to {}"; + LOGGER.debug(msg, id, path); + } + + private void updateCount(WorkloadInfo info) throws IOException { + int count = 0; + File file = new File(ARCHIVE_DIR, ".meta"); + String workloadId = null; + workloadId = info.getId(); + try { + count = parseID(workloadId); + } catch (Exception e) { + LOGGER.error("cannot parse workloadId", e); + } + + Writer writer = new BufferedWriter(new FileWriter(file)); + try { + writer.write(String.valueOf(count)); + } finally { + writer.close(); + } + LOGGER.debug("workload count has been updated as {}", count); + } + + @Override + public synchronized int getTotalWorkloads() { + int count = 0; + try { + count = retrieveCount(); + } catch (Exception e) { + LOGGER.error("fail to retrieve workload count", e); + } + return count; + } + + private int retrieveCount() throws IOException { + File file = new File(ARCHIVE_DIR, ".meta"); + if (!file.exists() || file.length() == 0) + return 0; + int count; + Reader reader = new BufferedReader(new FileReader(file)); + try { + count = new Scanner(reader).nextInt(); + } finally { + reader.close(); + } + LOGGER.debug("workload count has been retrieved as {}", count); + return count; + } + + private int parseID(String workloadID) throws NumberFormatException, IndexOutOfBoundsException { + int count = 0; + try { + count = Integer.parseInt(workloadID.substring(1)); + } catch (Exception e) { + LOGGER.error("cannot extract count from workloadId", e); + } + return count; + } + +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/archiver/WorkloadArchiver.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/archiver/WorkloadArchiver.java index 0015c633..cb3dec5a 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/archiver/WorkloadArchiver.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/archiver/WorkloadArchiver.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.archiver; @@ -24,9 +24,9 @@ /** * The interface for workload archiver, which provides possibility to archive * data to different target like file system, database... - * + * * @author ywang19, qzheng7 - * + * */ public interface WorkloadArchiver { diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractRunLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractRunLoader.java index acae391b..3a1694c4 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractRunLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractRunLoader.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.controller.loader; @@ -24,31 +24,31 @@ /** * This class is the base class for exporting run information. - * + * * @author ywang19, qzheng7 - * + * */ abstract class AbstractRunLoader implements RunLoader{ - protected BufferedReader reader; - - public AbstractRunLoader() { - /* empty */ - } - - @Override - public void init(BufferedReader reader) throws IOException{ - this.reader = reader; - readHeader(); - } - - @Override - public List load() throws IOException{ - return readWorkload(); - } - - protected abstract void readHeader() throws IOException; - - protected abstract List readWorkload() throws IOException; + protected BufferedReader reader; + + public AbstractRunLoader() { + /* empty */ + } + + @Override + public void init(BufferedReader reader) throws IOException{ + this.reader = reader; + readHeader(); + } + + @Override + public List load() throws IOException{ + return readWorkload(); + } + + protected abstract void readHeader() throws IOException; + + protected abstract List readWorkload() throws IOException; } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractSnapshotLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractSnapshotLoader.java index 0a634c76..1a1b2ea6 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractSnapshotLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractSnapshotLoader.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.controller.loader; @@ -24,36 +24,36 @@ /** * This class is the base class for exporting run information. - * + * * @author ywang19, qzheng7 - * + * */ abstract class AbstractSnapshotLoader implements SnapshotLoader{ - protected BufferedReader reader; - protected WorkloadInfo workloadContext; - protected StageInfo stageContext; - - public AbstractSnapshotLoader() { - /* empty */ - } - - @Override - public void init(BufferedReader reader, WorkloadInfo workloadContext, - String stageId) throws IOException { - this.reader = reader; - this.workloadContext = workloadContext; - this.stageContext = workloadContext.getStageInfo(stageId); - } - - @Override - public void load() throws IOException{ - readHeader(); - readSnapshot(); - } - - protected abstract void readHeader() throws IOException; - - protected abstract void readSnapshot() throws IOException; + protected BufferedReader reader; + protected WorkloadInfo workloadContext; + protected StageInfo stageContext; + + public AbstractSnapshotLoader() { + /* empty */ + } + + @Override + public void init(BufferedReader reader, WorkloadInfo workloadContext, + String stageId) throws IOException { + this.reader = reader; + this.workloadContext = workloadContext; + this.stageContext = workloadContext.getStageInfo(stageId); + } + + @Override + public void load() throws IOException{ + readHeader(); + readSnapshot(); + } + + protected abstract void readHeader() throws IOException; + + protected abstract void readSnapshot() throws IOException; } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractWorkloadFileLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractWorkloadFileLoader.java index 753b544f..ebd25dcd 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractWorkloadFileLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/AbstractWorkloadFileLoader.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.controller.loader; @@ -23,33 +23,33 @@ /** * This class is the base class for exporting run information. - * + * * @author ywang19, qzheng7 - * + * */ abstract class AbstractWorkloadFileLoader implements WorkloadFileLoader{ - protected BufferedReader reader; - protected WorkloadInfo workloadContext; - - public AbstractWorkloadFileLoader() { - /* empty */ - } - - @Override - public void init(BufferedReader reader, WorkloadInfo workloadContext) throws IOException{ - this.reader = reader; - this.workloadContext = workloadContext; - readHeader(); - } - - @Override - public void load() throws IOException{ - readWorkload(); - } - - protected abstract void readHeader() throws IOException; - - protected abstract void readWorkload() throws IOException; + protected BufferedReader reader; + protected WorkloadInfo workloadContext; + + public AbstractWorkloadFileLoader() { + /* empty */ + } + + @Override + public void init(BufferedReader reader, WorkloadInfo workloadContext) throws IOException{ + this.reader = reader; + this.workloadContext = workloadContext; + readHeader(); + } + + @Override + public void load() throws IOException{ + readWorkload(); + } + + protected abstract void readHeader() throws IOException; + + protected abstract void readWorkload() throws IOException; } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVRunLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVRunLoader.java index d3c90cd7..b6d3de56 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVRunLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVRunLoader.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.loader; @@ -34,64 +34,64 @@ class CSVRunLoader extends AbstractRunLoader { - private List workloads = new ArrayList(); - + private List workloads = new ArrayList(); + public CSVRunLoader(BufferedReader reader) throws IOException { super.init(reader); } - - @Override - protected void readHeader() throws IOException { - this.reader.readLine(); - } - - @Override - protected List readWorkload() throws IOException { - String workloadRecordLine = null; - while ((workloadRecordLine = this.reader.readLine()) != null) { - String[] columns = workloadRecordLine.split(","); - WorkloadContext workloadContext = new WorkloadContext(); - workloadContext.setArchived(true); - workloadContext.setId(columns[0]); - Report report = new Report(); - workloadContext.setReport(report); - Workload workload = new Workload(); - workload.setName(columns[1]); - workloadContext.setWorkload(workload); - try { - if(!columns[2].isEmpty()) - workloadContext.setSubmitDate(DATETIME.parse(columns[2])); - if(!columns[3].isEmpty()) - workloadContext.setStartDate(DATETIME.parse(columns[3])); - if(!columns[4].isEmpty()) - workloadContext.setStopDate(DATETIME.parse(columns[4])); - } catch (ParseException e) { - e.printStackTrace(); - } - - workloadContext.setOpInfo(columns[5].split(" ")); - for (WorkloadState state : WorkloadState.values()){ - if(columns[6].equalsIgnoreCase(state.toString().toLowerCase())){ - workloadContext.setState(state); - break; - } - } - int index = 6; - while(++index <= columns.length-1) { - String str[] = columns[index].split("@"); - String stateName = str[0].trim(); - Date stateDate = null; - try { - if(str.length > 1) - stateDate = DATETIME.parse(str[1].trim()); - } catch (ParseException e) { - e.printStackTrace(); - } - workloadContext.setState(stateName, stateDate); - } - workloads.add(workloadContext); - } - return workloads; - } + + @Override + protected void readHeader() throws IOException { + this.reader.readLine(); + } + + @Override + protected List readWorkload() throws IOException { + String workloadRecordLine = null; + while ((workloadRecordLine = this.reader.readLine()) != null) { + String[] columns = workloadRecordLine.split(","); + WorkloadContext workloadContext = new WorkloadContext(); + workloadContext.setArchived(true); + workloadContext.setId(columns[0]); + Report report = new Report(); + workloadContext.setReport(report); + Workload workload = new Workload(); + workload.setName(columns[1]); + workloadContext.setWorkload(workload); + try { + if(!columns[2].isEmpty()) + workloadContext.setSubmitDate(DATETIME.parse(columns[2])); + if(!columns[3].isEmpty()) + workloadContext.setStartDate(DATETIME.parse(columns[3])); + if(!columns[4].isEmpty()) + workloadContext.setStopDate(DATETIME.parse(columns[4])); + } catch (ParseException e) { + e.printStackTrace(); + } + + workloadContext.setOpInfo(columns[5].split(" ")); + for (WorkloadState state : WorkloadState.values()){ + if(columns[6].equalsIgnoreCase(state.toString().toLowerCase())){ + workloadContext.setState(state); + break; + } + } + int index = 6; + while(++index <= columns.length-1) { + String str[] = columns[index].split("@"); + String stateName = str[0].trim(); + Date stateDate = null; + try { + if(str.length > 1) + stateDate = DATETIME.parse(str[1].trim()); + } catch (ParseException e) { + e.printStackTrace(); + } + workloadContext.setState(stateName, stateDate); + } + workloads.add(workloadContext); + } + return workloads; + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVSnapshotLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVSnapshotLoader.java index 23e94693..f17c781b 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVSnapshotLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVSnapshotLoader.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.controller.loader; @@ -36,96 +36,96 @@ class CSVSnapshotLoader extends AbstractSnapshotLoader { - private String[] name = null; - private int opNum = 0; - - public CSVSnapshotLoader(BufferedReader reader, - WorkloadInfo workloadContext, String stageId) throws IOException { - super.init(reader, workloadContext, stageId); - } - - @Override - protected void readHeader() throws IOException { - if(this.reader.readLine() == null){ - return; - } - String workloadRecordLine = this.reader.readLine(); - String[] columns = workloadRecordLine.split(","); - for (Work work : stageContext.getStage().getWorks()) { - opNum += work.getOperations().size(); - } - name = new String[opNum]; - for (int i = 0; i < opNum; i++) - name[i] = columns[i + 1]; - } - - @Override - protected void readSnapshot() throws IOException { - String workloadRecordLine = null; - while ((workloadRecordLine = this.reader.readLine()) != null) { - String[] columns = workloadRecordLine.split(","); - Date timestamp = null; - try { - timestamp = TIME.parse(columns[0]); - } catch (ParseException e) { - e.printStackTrace(); - } - Snapshot snapshot = new Snapshot(loadReport(columns), timestamp); - snapshot.setMinVersion(Integer.valueOf(columns[1 + opNum * 7])); - snapshot.setVersion(Integer.valueOf(columns[2 + opNum * 7])); - snapshot.setMaxVersion(Integer.valueOf(columns[3 + opNum * 7])); - stageContext.getSnapshotRegistry().addItem(snapshot); - } - } - - private Report loadReport(String[] columns) { - Report report = new Report(); - List metrics = loadMetrics(columns); - for (Metrics metric : metrics) - report.addMetrics(metric); - return report; - } - - private List loadMetrics(String[] columns) { - List metrics = new ArrayList(); - for (int i = 0; i < opNum; i++) { - Metrics metric = new Metrics(); - metric.setName(name[i]); - int n = name[i].lastIndexOf("-"); - if (n > 0) { - metric.setOpName(name[i].substring(0, n)); - metric.setSampleType(name[i].substring(n + 1)); - } else { - metric.setOpName(name[i]); - metric.setSampleType(name[i]); - } - metric.setSampleCount(getIntValue(columns[i + 1])); - metric.setByteCount(getLongValue(columns[i + opNum + 1])); - double rt = getDoubleValue(columns[i + opNum * 2 + 1]); - metric.setAvgResTime(rt); - double pt = getDoubleValue(columns[i + opNum * 3 + 1]); - metric.setAvgXferTime(rt - pt); - metric.setThroughput(getDoubleValue(columns[i + opNum * 4 + 1])); - metric.setBandwidth(getDoubleValue(columns[i + opNum * 5 + 1])); -// metric.setRatio(columns[i + opNum * 5 + 1].equalsIgnoreCase("N/A") ? 0D -// : Double.valueOf(columns[i + opNum * 5 + 1].substring(0, -// columns[i + opNum * 5 + 1].length() - 1)) / 100.0); - setRatio(columns[i + opNum * 6 + 1], metric); - metrics.add(metric); - } - return metrics; - } - - private void setRatio(String column, Metrics metrics) { - if (!column.equalsIgnoreCase("N/A")) { - metrics.setRatio(Double.valueOf(column.substring(0, - column.length() - 1)) / 100.0); - metrics.setTotalSampleCount(metrics.getSampleCount() - / metrics.getRatio() > Integer.MAX_VALUE ? Integer.MAX_VALUE - : (int) (metrics.getSampleCount() / metrics.getRatio())); - } else { - metrics.setRatio(0D); - metrics.setTotalSampleCount(0); - } - } -} + private String[] name = null; + private int opNum = 0; + + public CSVSnapshotLoader(BufferedReader reader, + WorkloadInfo workloadContext, String stageId) throws IOException { + super.init(reader, workloadContext, stageId); + } + + @Override + protected void readHeader() throws IOException { + if(this.reader.readLine() == null){ + return; + } + String workloadRecordLine = this.reader.readLine(); + String[] columns = workloadRecordLine.split(","); + for (Work work : stageContext.getStage().getWorks()) { + opNum += work.getOperations().size(); + } + name = new String[opNum]; + for (int i = 0; i < opNum; i++) + name[i] = columns[i + 1]; + } + + @Override + protected void readSnapshot() throws IOException { + String workloadRecordLine = null; + while ((workloadRecordLine = this.reader.readLine()) != null) { + String[] columns = workloadRecordLine.split(","); + Date timestamp = null; + try { + timestamp = TIME.parse(columns[0]); + } catch (ParseException e) { + e.printStackTrace(); + } + Snapshot snapshot = new Snapshot(loadReport(columns), timestamp); + snapshot.setMinVersion(Integer.valueOf(columns[1 + opNum * 7])); + snapshot.setVersion(Integer.valueOf(columns[2 + opNum * 7])); + snapshot.setMaxVersion(Integer.valueOf(columns[3 + opNum * 7])); + stageContext.getSnapshotRegistry().addItem(snapshot); + } + } + + private Report loadReport(String[] columns) { + Report report = new Report(); + List metrics = loadMetrics(columns); + for (Metrics metric : metrics) + report.addMetrics(metric); + return report; + } + + private List loadMetrics(String[] columns) { + List metrics = new ArrayList(); + for (int i = 0; i < opNum; i++) { + Metrics metric = new Metrics(); + metric.setName(name[i]); + int n = name[i].lastIndexOf("-"); + if (n > 0) { + metric.setOpName(name[i].substring(0, n)); + metric.setSampleType(name[i].substring(n + 1)); + } else { + metric.setOpName(name[i]); + metric.setSampleType(name[i]); + } + metric.setSampleCount(getIntValue(columns[i + 1])); + metric.setByteCount(getLongValue(columns[i + opNum + 1])); + double rt = getDoubleValue(columns[i + opNum * 2 + 1]); + metric.setAvgResTime(rt); + double pt = getDoubleValue(columns[i + opNum * 3 + 1]); + metric.setAvgXferTime(rt - pt); + metric.setThroughput(getDoubleValue(columns[i + opNum * 4 + 1])); + metric.setBandwidth(getDoubleValue(columns[i + opNum * 5 + 1])); +// metric.setRatio(columns[i + opNum * 5 + 1].equalsIgnoreCase("N/A") ? 0D +// : Double.valueOf(columns[i + opNum * 5 + 1].substring(0, +// columns[i + opNum * 5 + 1].length() - 1)) / 100.0); + setRatio(columns[i + opNum * 6 + 1], metric); + metrics.add(metric); + } + return metrics; + } + + private void setRatio(String column, Metrics metrics) { + if (!column.equalsIgnoreCase("N/A")) { + metrics.setRatio(Double.valueOf(column.substring(0, + column.length() - 1)) / 100.0); + metrics.setTotalSampleCount(metrics.getSampleCount() + / metrics.getRatio() > Integer.MAX_VALUE ? Integer.MAX_VALUE + : (int) (metrics.getSampleCount() / metrics.getRatio())); + } else { + metrics.setRatio(0D); + metrics.setTotalSampleCount(0); + } + } +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVWorkloadFileLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVWorkloadFileLoader.java index c2a74531..ff45ce69 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVWorkloadFileLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/CSVWorkloadFileLoader.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.controller.loader; @@ -31,130 +31,130 @@ class CSVWorkloadFileLoader extends AbstractWorkloadFileLoader { - public CSVWorkloadFileLoader(BufferedReader reader, - WorkloadInfo workloadContext) throws IOException { - super.init(reader, workloadContext); - } - - @Override - protected void readHeader() throws IOException { - this.reader.readLine(); - } - - @Override - protected void readWorkload() throws IOException { - String workloadRecordLine = null; - String lastStageName = null; - String stageId = null; - boolean sameStage = false; - int index = 1; - while ((workloadRecordLine = this.reader.readLine()) != null) { - String[] columns = workloadRecordLine.split(","); - sameStage = true; - if (lastStageName == null - || !lastStageName.equalsIgnoreCase(columns[0])) { - lastStageName = columns[0]; - stageId = "s" + index++; - sameStage = false; - } - if (columns[16].equalsIgnoreCase("completed")) { - Metrics metrics = loadMetrics(columns); - if (!sameStage) { - Report report = new Report(); - workloadContext.getStageInfo(stageId).setReport(report); - } - workloadContext.getStageInfo(stageId).getReport() - .addMetrics(metrics); - workloadContext.getReport().addMetrics(metrics); - } - for (StageState state : StageState.values()) { - if (columns[16] - .equalsIgnoreCase(state.toString().toLowerCase())) { - workloadContext.getStageInfo(stageId).setState(state, true); - break; - } - } - int pos = 16; - while (!sameStage && ++pos <= columns.length - 1) { - String str[] = columns[pos].split("@"); - String stateName = str[0].trim(); - Date stateDate = null; - try { - stateDate = DATETIME.parse(str[1].trim()); - } catch (ParseException e) { - e.printStackTrace(); - } - workloadContext.getStageInfo(stageId).setState(stateName, - stateDate); - } - } - } - - private Metrics loadMetrics(String[] columns) { - Metrics metrics = new Metrics(); - int n = columns[1].lastIndexOf("-"); - metrics.setName(columns[1]); - if (n > 0) { - metrics.setOpName(columns[1].substring(0, n)); - metrics.setSampleType(columns[1].substring(n + 1)); - } else { - metrics.setOpName(columns[1]); - metrics.setSampleType(columns[1]); - } - metrics.setOpType(columns[2]); - metrics.setSampleCount(Integer.valueOf(columns[3])); - metrics.setByteCount(Long.valueOf(columns[4])); - double rt = columns[5].equalsIgnoreCase("N/A") ? 0 : Double.valueOf(columns[5]); - metrics.setAvgResTime(rt); - double pt = columns[6].equalsIgnoreCase("N/A") ? 0 : Double.valueOf(columns[6]); - metrics.setAvgXferTime(rt - pt); - metrics.setLatency(loadHistogram(columns)); - metrics.setThroughput(Double.valueOf(columns[13])); - metrics.setBandwidth(Double.valueOf(columns[14])); - setRatio(columns[15], metrics); - return metrics; - } - - private void setRatio(String column, Metrics metrics) { - if (!column.equalsIgnoreCase("N/A")) { - metrics.setRatio(Double.valueOf(column.substring(0, - column.length() - 1)) / 100.0); - metrics.setTotalSampleCount(metrics.getSampleCount() - / metrics.getRatio() > Integer.MAX_VALUE ? Integer.MAX_VALUE - : (int) (metrics.getSampleCount() / metrics.getRatio())); - } else { - metrics.setRatio(0D); - metrics.setTotalSampleCount(0); - } - } - - private Histogram loadHistogram(String[] columns) { - Histogram histogram = new Histogram(); - long[] l_60 = new long[2]; - l_60[1] = columns[7].equalsIgnoreCase("N/A") ? 0L : Long - .valueOf(columns[7]); - histogram.set_60(l_60); - long[] l_80 = new long[2]; - l_80[1] = columns[8].equalsIgnoreCase("N/A") ? 0L : Long - .valueOf(columns[8]); - histogram.set_80(l_80); - long[] l_90 = new long[2]; - l_90[1] = columns[9].equalsIgnoreCase("N/A") ? 0L : Long - .valueOf(columns[9]); - histogram.set_90(l_90); - long[] l_95 = new long[2]; - l_95[1] = columns[10].equalsIgnoreCase("N/A") ? 0L : Long - .valueOf(columns[10]); - histogram.set_95(l_95); - long[] l_99 = new long[2]; - l_99[1] = columns[11].equalsIgnoreCase("N/A") ? 0L : Long - .valueOf(columns[11]); - histogram.set_99(l_99); - long[] l_100 = new long[2]; - l_100[1] = columns[12].equalsIgnoreCase("N/A") ? 0L : Long - .valueOf(columns[12]); - histogram.set_100(l_100); - return histogram; - } + public CSVWorkloadFileLoader(BufferedReader reader, + WorkloadInfo workloadContext) throws IOException { + super.init(reader, workloadContext); + } + + @Override + protected void readHeader() throws IOException { + this.reader.readLine(); + } + + @Override + protected void readWorkload() throws IOException { + String workloadRecordLine = null; + String lastStageName = null; + String stageId = null; + boolean sameStage = false; + int index = 1; + while ((workloadRecordLine = this.reader.readLine()) != null) { + String[] columns = workloadRecordLine.split(","); + sameStage = true; + if (lastStageName == null + || !lastStageName.equalsIgnoreCase(columns[0])) { + lastStageName = columns[0]; + stageId = "s" + index++; + sameStage = false; + } + if (columns[16].equalsIgnoreCase("completed")) { + Metrics metrics = loadMetrics(columns); + if (!sameStage) { + Report report = new Report(); + workloadContext.getStageInfo(stageId).setReport(report); + } + workloadContext.getStageInfo(stageId).getReport() + .addMetrics(metrics); + workloadContext.getReport().addMetrics(metrics); + } + for (StageState state : StageState.values()) { + if (columns[16] + .equalsIgnoreCase(state.toString().toLowerCase())) { + workloadContext.getStageInfo(stageId).setState(state, true); + break; + } + } + int pos = 16; + while (!sameStage && ++pos <= columns.length - 1) { + String str[] = columns[pos].split("@"); + String stateName = str[0].trim(); + Date stateDate = null; + try { + stateDate = DATETIME.parse(str[1].trim()); + } catch (ParseException e) { + e.printStackTrace(); + } + workloadContext.getStageInfo(stageId).setState(stateName, + stateDate); + } + } + } + + private Metrics loadMetrics(String[] columns) { + Metrics metrics = new Metrics(); + int n = columns[1].lastIndexOf("-"); + metrics.setName(columns[1]); + if (n > 0) { + metrics.setOpName(columns[1].substring(0, n)); + metrics.setSampleType(columns[1].substring(n + 1)); + } else { + metrics.setOpName(columns[1]); + metrics.setSampleType(columns[1]); + } + metrics.setOpType(columns[2]); + metrics.setSampleCount(Integer.valueOf(columns[3])); + metrics.setByteCount(Long.valueOf(columns[4])); + double rt = columns[5].equalsIgnoreCase("N/A") ? 0 : Double.valueOf(columns[5]); + metrics.setAvgResTime(rt); + double pt = columns[6].equalsIgnoreCase("N/A") ? 0 : Double.valueOf(columns[6]); + metrics.setAvgXferTime(rt - pt); + metrics.setLatency(loadHistogram(columns)); + metrics.setThroughput(Double.valueOf(columns[13])); + metrics.setBandwidth(Double.valueOf(columns[14])); + setRatio(columns[15], metrics); + return metrics; + } + + private void setRatio(String column, Metrics metrics) { + if (!column.equalsIgnoreCase("N/A")) { + metrics.setRatio(Double.valueOf(column.substring(0, + column.length() - 1)) / 100.0); + metrics.setTotalSampleCount(metrics.getSampleCount() + / metrics.getRatio() > Integer.MAX_VALUE ? Integer.MAX_VALUE + : (int) (metrics.getSampleCount() / metrics.getRatio())); + } else { + metrics.setRatio(0D); + metrics.setTotalSampleCount(0); + } + } + + private Histogram loadHistogram(String[] columns) { + Histogram histogram = new Histogram(); + long[] l_60 = new long[2]; + l_60[1] = columns[7].equalsIgnoreCase("N/A") ? 0L : Long + .valueOf(columns[7]); + histogram.set_60(l_60); + long[] l_80 = new long[2]; + l_80[1] = columns[8].equalsIgnoreCase("N/A") ? 0L : Long + .valueOf(columns[8]); + histogram.set_80(l_80); + long[] l_90 = new long[2]; + l_90[1] = columns[9].equalsIgnoreCase("N/A") ? 0L : Long + .valueOf(columns[9]); + histogram.set_90(l_90); + long[] l_95 = new long[2]; + l_95[1] = columns[10].equalsIgnoreCase("N/A") ? 0L : Long + .valueOf(columns[10]); + histogram.set_95(l_95); + long[] l_99 = new long[2]; + l_99[1] = columns[11].equalsIgnoreCase("N/A") ? 0L : Long + .valueOf(columns[11]); + histogram.set_99(l_99); + long[] l_100 = new long[2]; + l_100[1] = columns[12].equalsIgnoreCase("N/A") ? 0L : Long + .valueOf(columns[12]); + histogram.set_100(l_100); + return histogram; + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/Formats.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/Formats.java index 448335e5..6aa8f89c 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/Formats.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/Formats.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.loader; @@ -21,7 +21,7 @@ /** * This class defines necessary data/number formats. - * + * * @author ywang19, qzheng7 * */ @@ -37,17 +37,17 @@ class Formats { public static final NumberFormat NUM = new DecimalFormat("0.##"); public static final NumberFormat RATIO = new DecimalFormat("0.##%"); - + public static double getDoubleValue(String column) { - return column.equalsIgnoreCase("N/A") ? 0D : Double.valueOf(column); - } - - public static int getIntValue(String column) { - return column.equalsIgnoreCase("N/A") ? 0 : Integer.valueOf(column); - } - - public static long getLongValue(String column) { - return column.equalsIgnoreCase("N/A") ? 0L : Long.valueOf(column); - } + return column.equalsIgnoreCase("N/A") ? 0D : Double.valueOf(column); + } + + public static int getIntValue(String column) { + return column.equalsIgnoreCase("N/A") ? 0 : Integer.valueOf(column); + } + + public static long getLongValue(String column) { + return column.equalsIgnoreCase("N/A") ? 0L : Long.valueOf(column); + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/Loaders.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/Loaders.java index 4d412ffd..d63093d7 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/Loaders.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/Loaders.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.loader; @@ -24,30 +24,30 @@ /** * This class is a wrapper to construct different exporters by providing static construction methods. - * + * * @author ywang19, qzheng7 * */ public class Loaders { - public static RunLoader newRunExporter(BufferedReader reader) - throws IOException { - AbstractRunLoader loader = new CSVRunLoader(reader); - return loader; - } - - public static WorkloadFileLoader newWorkloadLoader(BufferedReader reader, - WorkloadInfo workloadContext) throws IOException { - AbstractWorkloadFileLoader loader = new CSVWorkloadFileLoader(reader, - workloadContext); - return loader; - } - - public static SnapshotLoader newSnapshotLoader(BufferedReader reader, - WorkloadInfo workload, String stageId) throws IOException { - AbstractSnapshotLoader loader = new CSVSnapshotLoader(reader, workload, - stageId); - return loader; - } + public static RunLoader newRunExporter(BufferedReader reader) + throws IOException { + AbstractRunLoader loader = new CSVRunLoader(reader); + return loader; + } + + public static WorkloadFileLoader newWorkloadLoader(BufferedReader reader, + WorkloadInfo workloadContext) throws IOException { + AbstractWorkloadFileLoader loader = new CSVWorkloadFileLoader(reader, + workloadContext); + return loader; + } + + public static SnapshotLoader newSnapshotLoader(BufferedReader reader, + WorkloadInfo workload, String stageId) throws IOException { + AbstractSnapshotLoader loader = new CSVSnapshotLoader(reader, workload, + stageId); + return loader; + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/RunLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/RunLoader.java index 09bb107d..24680a46 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/RunLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/RunLoader.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.loader; import java.io.BufferedReader; @@ -8,8 +26,8 @@ public interface RunLoader { - public void init(BufferedReader reader) throws IOException; + public void init(BufferedReader reader) throws IOException; - public List load() throws IOException; + public List load() throws IOException; } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/SimpleWorkloadLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/SimpleWorkloadLoader.java index 23243a2c..f51ec134 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/SimpleWorkloadLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/SimpleWorkloadLoader.java @@ -1,154 +1,172 @@ -package com.intel.cosbench.controller.loader; - -import java.io.*; -import java.util.ArrayList; -import java.util.List; - -import com.intel.cosbench.config.Stage; -import com.intel.cosbench.config.WorkloadResolver; -import com.intel.cosbench.config.XmlConfig; -import com.intel.cosbench.config.castor.CastorConfigTools; -import com.intel.cosbench.controller.model.StageContext; -import com.intel.cosbench.controller.model.StageRegistry; -import com.intel.cosbench.controller.model.WorkloadContext; -import com.intel.cosbench.log.LogFactory; -import com.intel.cosbench.log.Logger; -import com.intel.cosbench.model.StageInfo; -import com.intel.cosbench.model.StageState; -import com.intel.cosbench.model.WorkloadInfo; -import com.intel.cosbench.service.WorkloadLoader; - -public class SimpleWorkloadLoader implements WorkloadLoader { - private static final Logger LOGGER = LogFactory.getSystemLogger(); - - private static File ARCHIVE_DIR = new File("archive"); - -// static { -// if (!ROOT_DIR.exists()) -// ROOT_DIR.mkdirs(); -// String path = ROOT_DIR.getAbsolutePath(); -// LOGGER.info("using {} for loading workload archives", path); -// } - - public SimpleWorkloadLoader() { - this("archive"); - } - - public SimpleWorkloadLoader(final String archive) { - ARCHIVE_DIR = new File(archive); - - if (!ARCHIVE_DIR.exists()) - ARCHIVE_DIR.mkdirs(); - String path = ARCHIVE_DIR.getAbsolutePath(); - LOGGER.info("loading workload archives from {}", path); - } - - private static String getRunDirName(WorkloadInfo info) { - String name = info.getId(); - name += '-' + info.getWorkload().getName(); - return name; - } - - @Override - public List loadWorkloadRun() throws IOException { - File file = new File(ARCHIVE_DIR, "run-history.csv"); - if (!file.exists()) - return null; - BufferedReader reader = new BufferedReader(new FileReader(file)); - RunLoader loader = Loaders.newRunExporter(reader); - List workloads = new ArrayList(); - workloads = loader.load(); - if (reader != null) - reader.close(); - return workloads; - } - - @Override - public void loadWorkloadPageInfo(WorkloadInfo workloadContext) - throws IOException { - loadWorkloadConfig(workloadContext); - loadWorkloadFile(workloadContext); - } - - private void loadWorkloadConfig(WorkloadInfo workloadContext) - throws FileNotFoundException { - XmlConfig config = getWorkloadConfg(workloadContext); - if(config != null) { - WorkloadResolver resolver = CastorConfigTools.getWorkloadResolver(); - workloadContext.setWorkload(resolver.toWorkload(config)); - createStages(workloadContext); - }else { - ((WorkloadContext) workloadContext).setStageRegistry(new StageRegistry()); - } - } - - public static XmlConfig getWorkloadConfg(WorkloadInfo workloadContext) - throws FileNotFoundException { - File file = new File( - new File(ARCHIVE_DIR, getRunDirName(workloadContext)), - "workload-config.xml"); - if (!file.exists()) - return null; - XmlConfig config = new XmlConfig(new FileInputStream(file)); - return config; - } - - private void createStages(WorkloadInfo workloadContext) { - StageRegistry registry = new StageRegistry(); - int index = 1; - for (Stage stage : workloadContext.getWorkload().getWorkflow()) { - String id = "s" + index++; - registry.addStage(createStageContext(id, stage)); - } - ((WorkloadContext) workloadContext).setStageRegistry(registry); - } - - private static StageContext createStageContext(String id, Stage stage) { - StageContext context = new StageContext(); - context.setId(id); - context.setStage(stage); - context.setState(StageState.COMPLETED, true); - return context; - } - - private static String getWorkloadFileName(WorkloadInfo info) { - String name = info.getId(); - name += "-" + info.getWorkload().getName(); - return name; - } - - private void loadWorkloadFile(WorkloadInfo workloadContext) - throws IOException { - File file = new File( - new File(ARCHIVE_DIR, getRunDirName(workloadContext)), - getWorkloadFileName(workloadContext) + ".csv"); - if (!file.exists()) - return; - BufferedReader reader = new BufferedReader(new FileReader(file)); - WorkloadFileLoader loader = Loaders.newWorkloadLoader(reader, - workloadContext); - loader.load(); - } - - private static String getStageFileName(StageInfo info) { - String name = info.getId(); - name += "-" + info.getStage().getName(); - return name; - } - - @Override - public void loadStagePageInfo(WorkloadInfo workloadContext, String stageId) - throws IOException { - File file = new File( - new File(ARCHIVE_DIR, getRunDirName(workloadContext)), - getStageFileName(workloadContext.getStageInfo(stageId)) - + ".csv"); - if (!file.exists()) - return; - BufferedReader reader = new BufferedReader(new FileReader(file)); - SnapshotLoader loader = Loaders.newSnapshotLoader(reader, - workloadContext, stageId); - loader.load(); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.controller.loader; + +import java.io.*; +import java.util.ArrayList; +import java.util.List; + +import com.intel.cosbench.config.Stage; +import com.intel.cosbench.config.WorkloadResolver; +import com.intel.cosbench.config.XmlConfig; +import com.intel.cosbench.config.castor.CastorConfigTools; +import com.intel.cosbench.controller.model.StageContext; +import com.intel.cosbench.controller.model.StageRegistry; +import com.intel.cosbench.controller.model.WorkloadContext; +import com.intel.cosbench.log.LogFactory; +import com.intel.cosbench.log.Logger; +import com.intel.cosbench.model.StageInfo; +import com.intel.cosbench.model.StageState; +import com.intel.cosbench.model.WorkloadInfo; +import com.intel.cosbench.service.WorkloadLoader; + +public class SimpleWorkloadLoader implements WorkloadLoader { + private static final Logger LOGGER = LogFactory.getSystemLogger(); + + private static File ARCHIVE_DIR = new File("archive"); + +// static { +// if (!ROOT_DIR.exists()) +// ROOT_DIR.mkdirs(); +// String path = ROOT_DIR.getAbsolutePath(); +// LOGGER.info("using {} for loading workload archives", path); +// } + + public SimpleWorkloadLoader() { + this("archive"); + } + + public SimpleWorkloadLoader(final String archive) { + ARCHIVE_DIR = new File(archive); + + if (!ARCHIVE_DIR.exists()) + ARCHIVE_DIR.mkdirs(); + String path = ARCHIVE_DIR.getAbsolutePath(); + LOGGER.info("loading workload archives from {}", path); + } + + private static String getRunDirName(WorkloadInfo info) { + String name = info.getId(); + name += '-' + info.getWorkload().getName(); + return name; + } + + @Override + public List loadWorkloadRun() throws IOException { + File file = new File(ARCHIVE_DIR, "run-history.csv"); + if (!file.exists()) + return null; + BufferedReader reader = new BufferedReader(new FileReader(file)); + RunLoader loader = Loaders.newRunExporter(reader); + List workloads = new ArrayList(); + workloads = loader.load(); + if (reader != null) + reader.close(); + return workloads; + } + + @Override + public void loadWorkloadPageInfo(WorkloadInfo workloadContext) + throws IOException { + loadWorkloadConfig(workloadContext); + loadWorkloadFile(workloadContext); + } + + private void loadWorkloadConfig(WorkloadInfo workloadContext) + throws FileNotFoundException { + XmlConfig config = getWorkloadConfg(workloadContext); + if(config != null) { + WorkloadResolver resolver = CastorConfigTools.getWorkloadResolver(); + workloadContext.setWorkload(resolver.toWorkload(config)); + createStages(workloadContext); + }else { + ((WorkloadContext) workloadContext).setStageRegistry(new StageRegistry()); + } + } + + public static XmlConfig getWorkloadConfg(WorkloadInfo workloadContext) + throws FileNotFoundException { + File file = new File( + new File(ARCHIVE_DIR, getRunDirName(workloadContext)), + "workload-config.xml"); + if (!file.exists()) + return null; + XmlConfig config = new XmlConfig(new FileInputStream(file)); + return config; + } + + private void createStages(WorkloadInfo workloadContext) { + StageRegistry registry = new StageRegistry(); + int index = 1; + for (Stage stage : workloadContext.getWorkload().getWorkflow()) { + String id = "s" + index++; + registry.addStage(createStageContext(id, stage)); + } + ((WorkloadContext) workloadContext).setStageRegistry(registry); + } + + private static StageContext createStageContext(String id, Stage stage) { + StageContext context = new StageContext(); + context.setId(id); + context.setStage(stage); + context.setState(StageState.COMPLETED, true); + return context; + } + + private static String getWorkloadFileName(WorkloadInfo info) { + String name = info.getId(); + name += "-" + info.getWorkload().getName(); + return name; + } + + private void loadWorkloadFile(WorkloadInfo workloadContext) + throws IOException { + File file = new File( + new File(ARCHIVE_DIR, getRunDirName(workloadContext)), + getWorkloadFileName(workloadContext) + ".csv"); + if (!file.exists()) + return; + BufferedReader reader = new BufferedReader(new FileReader(file)); + WorkloadFileLoader loader = Loaders.newWorkloadLoader(reader, + workloadContext); + loader.load(); + } + + private static String getStageFileName(StageInfo info) { + String name = info.getId(); + name += "-" + info.getStage().getName(); + return name; + } + + @Override + public void loadStagePageInfo(WorkloadInfo workloadContext, String stageId) + throws IOException { + File file = new File( + new File(ARCHIVE_DIR, getRunDirName(workloadContext)), + getStageFileName(workloadContext.getStageInfo(stageId)) + + ".csv"); + if (!file.exists()) + return; + BufferedReader reader = new BufferedReader(new FileReader(file)); + SnapshotLoader loader = Loaders.newSnapshotLoader(reader, + workloadContext, stageId); + loader.load(); + } + +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/SnapshotLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/SnapshotLoader.java index 0b411372..7a0b5704 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/SnapshotLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/SnapshotLoader.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.loader; import java.io.BufferedReader; @@ -7,9 +25,9 @@ public interface SnapshotLoader { - void init(BufferedReader reader, WorkloadInfo workloadContext, - String stageId) throws IOException; - - public void load() throws IOException; + void init(BufferedReader reader, WorkloadInfo workloadContext, + String stageId) throws IOException; + + public void load() throws IOException; } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/WorkloadFileLoader.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/WorkloadFileLoader.java index 2afc95d8..768683ee 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/WorkloadFileLoader.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/loader/WorkloadFileLoader.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.loader; import java.io.BufferedReader; @@ -7,9 +25,9 @@ public interface WorkloadFileLoader { - void init(BufferedReader reader, WorkloadInfo workloadContext) throws IOException; - - public void load() throws IOException; + void init(BufferedReader reader, WorkloadInfo workloadContext) throws IOException; + + public void load() throws IOException; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ControllerContext.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ControllerContext.java index 179e2bec..fb838981 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ControllerContext.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ControllerContext.java @@ -1,92 +1,112 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.controller.model; - -import com.intel.cosbench.model.*; - -/** - * This class encapsulates the configurations in controller.conf. - * - * @author ywang19, qzheng7 - * - */ -public class ControllerContext implements ControllerInfo { - - private String name; - private String url; - private String archive_dir; - private int concurrency; - private DriverRegistry driverRegistry; - - public ControllerContext() { - /* empty */ - } - - @Override - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - @Override - public String getUrl() { - return url; - } - - public void setUrl(String url) { - this.url = url; - } - - public String getArchive_dir() { - return archive_dir; - } - - public void setArchive_dir(String archive_dir) { - this.archive_dir = archive_dir; - } - - public int getConcurrency() { - return concurrency; - } - - public void setConcurrency(int concurrency) { - this.concurrency = concurrency; - } - - public DriverRegistry getDriverRegistry() { - return driverRegistry; - } - - public void setDriverRegistry(DriverRegistry driverRegistry) { - this.driverRegistry = driverRegistry; - } - - @Override - public int getDriverCount() { - return driverRegistry.getSize(); - } - - @Override - public DriverInfo[] getDriverInfos() { - return driverRegistry.getAllDrivers(); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.controller.model; + +import java.util.Date; +import com.intel.cosbench.model.*; + +/** + * This class encapsulates the configurations in controller.conf. + * + * @author ywang19, qzheng7 + * + */ +public class ControllerContext implements ControllerInfo { + + private String name; + private String url; + private String archive_dir; + private int concurrency; + private DriverRegistry driverRegistry; + private String version; + private String time; + + public String getTime() { + time = new Date().toString(); + return time; + } + + public void setTime(String timeStr) { + time = timeStr; + } + + public String getVersion() { + return version; + } + + public void setVersion(String version) { + this.version = version; + } + + public ControllerContext() { + /* empty */ + } + + @Override + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + @Override + public String getUrl() { + return url; + } + + public void setUrl(String url) { + this.url = url; + } + + public String getArchive_dir() { + return archive_dir; + } + + public void setArchive_dir(String archive_dir) { + this.archive_dir = archive_dir; + } + + public int getConcurrency() { + return concurrency; + } + + public void setConcurrency(int concurrency) { + this.concurrency = concurrency; + } + + public DriverRegistry getDriverRegistry() { + return driverRegistry; + } + + public void setDriverRegistry(DriverRegistry driverRegistry) { + this.driverRegistry = driverRegistry; + } + + @Override + public int getDriverCount() { + return driverRegistry.getSize(); + } + + @Override + public DriverInfo[] getDriverInfos() { + return driverRegistry.getAllDrivers(); + } + +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/DriverContext.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/DriverContext.java index c93e5c0c..e2bb2d36 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/DriverContext.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/DriverContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; @@ -25,21 +25,22 @@ /** * This class encapsulates the driver sections in controller.conf. - * + * * @author ywang19, qzheng7 - * + * */ public class DriverContext implements DriverInfo, MapRegistry.Item { private String name; private String url; private boolean aliveState; + // pIDMap - private Map pidMap = new HashMap(); - // logMap<'wId'+'sId', ScriptLog> - private Map scriptsLog = new HashMap(); + private Map pidMap = new HashMap(); + // logMap<'wId'+'sId', ScriptLog> + private Map scriptsLog = new HashMap(); - public DriverContext() { + public DriverContext() { /* empty */ } @@ -68,33 +69,39 @@ public void setAliveState(boolean aliveState) { @Override public boolean getAliveState(){ - return aliveState; + return aliveState; + } + + public String getPidMapValue(String scriptName) { + String pid = pidMap.remove(scriptName); + return (pid == null) ? "0" : pid; + } + + public void putPidMap(String scriptName, String pid) { + if (pid == null) + pidMap.put(scriptName, "0"); + pidMap.put(scriptName, pid); + } + + @Override + public Map getLogMap() { + return scriptsLog; + } + + public void putLogMap(String wsId, String ScriptLog) { + if (wsId == null || wsId.isEmpty()) + return; + scriptsLog.put(wsId, ScriptLog); + } + + public String getLogMapValue(String wsId) { + return scriptsLog.remove(wsId); + } + + @Override + public String getVersion() { + // TODO Auto-generated method stub + return null; } - public String getPidMapValue(String scriptName) { - String pid = pidMap.remove(scriptName); - return (pid == null) ? "0" : pid; - } - - public void putPidMap(String scriptName, String pid) { - if (pid == null) - pidMap.put(scriptName, "0"); - pidMap.put(scriptName, pid); - } - - @Override - public Map getLogMap() { - return scriptsLog; - } - - public void putLogMap(String wsId, String ScriptLog) { - if (wsId == null || wsId.isEmpty()) - return; - scriptsLog.put(wsId, ScriptLog); - } - - public String getLogMapValue(String wsId) { - return scriptsLog.remove(wsId); - } - } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/DriverRegistry.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/DriverRegistry.java index c119040a..d6953082 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/DriverRegistry.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/DriverRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ErrorSummary.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ErrorSummary.java index e16cbb4c..4530327e 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ErrorSummary.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ErrorSummary.java @@ -1,24 +1,42 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.model; import java.util.HashMap; public class ErrorSummary { - private HashMap errorCodeAndNum; - public ErrorSummary(){ - errorCodeAndNum = new HashMap(); - } - public ErrorSummary(HashMap errors){ - errorCodeAndNum = errors; - } - public HashMap getErrorCodeAndNum() { - return errorCodeAndNum; - } - public void assignEntry(String key, Integer value){ - errorCodeAndNum.put(key, value); - } - public boolean contains(String key){ - return errorCodeAndNum.containsKey(key); - } - + private HashMap errorCodeAndNum; + public ErrorSummary(){ + errorCodeAndNum = new HashMap(); + } + public ErrorSummary(HashMap errors){ + errorCodeAndNum = errors; + } + public HashMap getErrorCodeAndNum() { + return errorCodeAndNum; + } + public void assignEntry(String key, Integer value){ + errorCodeAndNum.put(key, value); + } + public boolean contains(String key){ + return errorCodeAndNum.containsKey(key); + } + } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/SchedulePlan.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/SchedulePlan.java index 9654814b..911dcc21 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/SchedulePlan.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/SchedulePlan.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; @@ -23,9 +23,9 @@ /** * The class represents one schedule plan, one schedule plan depends on a few * factors like the driver, work, total worker count, current worker number... - * + * * @author ywang19, qzheng7 - * + * */ public class SchedulePlan implements ScheduleInfo { diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ScheduleRegistry.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ScheduleRegistry.java index a1471123..99e3b34e 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ScheduleRegistry.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/ScheduleRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/SnapshotRegistry.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/SnapshotRegistry.java index 07ab200a..2c0a04c1 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/SnapshotRegistry.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/SnapshotRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageContext.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageContext.java index c081c7a4..7f61a426 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageContext.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; @@ -26,9 +26,9 @@ /** * This class encapsulates one work stage. - * + * * @author ywang19, qzheng7 - * + * */ public class StageContext implements StageInfo { @@ -47,6 +47,16 @@ public class StageContext implements StageInfo { private transient List listeners = new ArrayList(); + private List taskReports = new ArrayList(); + + public List getTaskReports() { + return taskReports; + } + + public void setTaskReports(List taskReports) { + this.taskReports = taskReports; + } + public StageContext() { /* empty */ } @@ -65,21 +75,21 @@ public StageState getState() { } public void setState(StageState state) { - setState(state, false); + setState(state, false); } - + public void setState(StageState state, boolean archived) { this.state = state; - if(archived) - return; + if(archived) + return; stateHistory.addState(state.name()); if (StageState.isStopped(state)) fireStageStopped(); } - + @Override public void setState(String state, Date date) { - stateHistory.addState(state,date); + stateHistory.addState(state,date); } private void fireStageStopped() { @@ -93,8 +103,14 @@ public Report mergeReport() { if (taskRegistry == null) return new Report(); ReportMerger merger = new ReportMerger(); - for (TaskContext task : taskRegistry) + for (TaskContext task : taskRegistry){ + TaskReport tReport=new TaskReport(); + tReport.setReport(task.getReport()); + tReport.setDriverName(task.getSchedule().getDriver().getName()); + tReport.setDriverUrl(task.getSchedule().getDriver().getUrl()); + taskReports.add(tReport); merger.add(task.getReport()); + } return merger.merge(); } @@ -217,9 +233,9 @@ public void disposeRuntime() { listeners = null; } - @Override - public ListRegistry getSnapshotRegistry() { - return snapshotRegistry; - } + @Override + public ListRegistry getSnapshotRegistry() { + return snapshotRegistry; + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageListener.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageListener.java index 36eb2b53..428669bc 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageListener.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageListener.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageRegistry.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageRegistry.java index 028f5b65..4a309887 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageRegistry.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/StageRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/TaskContext.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/TaskContext.java index 767a786f..9dbc55e6 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/TaskContext.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/TaskContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,11 +12,13 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; +import java.awt.List; +import java.util.ArrayList; import java.util.HashMap; import org.apache.http.client.HttpClient; @@ -43,10 +45,19 @@ public class TaskContext implements TaskInfo { private transient volatile Report report = new Report(); /* Each task starts with an empty log */ private volatile transient String log = "[N/A]"; - + /* Each task starts with an empty error statistics log*/ private volatile transient HashMap errorStatistics = new HashMap(); - + /*Each task for workers report starts with empty*/ + private ArrayList wrReport = new ArrayList(); + + public ArrayList getWrReport() { + return wrReport; + } + + public void setWrReport(ArrayList wrReport) { + this.wrReport = wrReport; + } public TaskContext() { /* empty */ @@ -132,25 +143,25 @@ public String getLog() { public void setLog(String log) { this.log = log; } - + public HashMap getErrorStatistics() { - return errorStatistics; - } + return errorStatistics; + } - public void setErrorStatistics(HashMap errorStatistics) { - this.errorStatistics = errorStatistics; - } + public void setErrorStatistics(HashMap errorStatistics) { + this.errorStatistics = errorStatistics; + } - @Override + @Override public void disposeRuntime() { - if(TaskState.isStopped(state)) { - httpClient = null; - mapper = null; - report = null; - log = null; - snapshot = new Snapshot(); - } + if(TaskState.isStopped(state)) { + httpClient = null; + mapper = null; + report = null; + log = null; + snapshot = new Snapshot(); + } } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/TaskRegistry.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/TaskRegistry.java index 213cac60..eff84bf2 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/TaskRegistry.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/TaskRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/WorkloadContext.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/WorkloadContext.java index 1debe4a5..9994df38 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/WorkloadContext.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/WorkloadContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; @@ -22,11 +22,13 @@ import com.intel.cosbench.bench.*; import com.intel.cosbench.config.*; +import com.intel.cosbench.log.LogFactory; import com.intel.cosbench.log.Logger; import com.intel.cosbench.model.*; public class WorkloadContext implements WorkloadInfo { + private static final Logger LOGGER = LogFactory.getSystemLogger(); private String id; private Date submitDate; private Date startDate; @@ -46,11 +48,11 @@ public class WorkloadContext implements WorkloadInfo { private volatile Report report = null; // will be merged from stage reports private transient List listeners = new ArrayList(); - + private String[] opInfo; - + private boolean archived = false; - + // private HashMap> errorStatistics = new HashMap>(); private HashMap errorStatistics = new HashMap(); public WorkloadContext() { @@ -65,13 +67,13 @@ public String getId() { public void setId(String id) { this.id = id; } - + public int getOrder(){ - return this.order; + return this.order; } - + public void setOrder(int order){ - this.order = order; + this.order = order; } @Override @@ -108,32 +110,32 @@ public WorkloadState getState() { public void setState(WorkloadState state) { this.state = state; - if(this.archived) - return; + if(this.archived) + return; stateHistory.addState(state.name()); if (WorkloadState.isRunning(state)) fireWorkloadStarted(); if (WorkloadState.isStopped(state)) fireWorkloadStopped(); } - + public void setState(String state, Date date){ - stateHistory.addState(state, date); + stateHistory.addState(state, date); } private void fireWorkloadStarted() { for (WorkloadListener listener : listeners) listener.workloadStarted(this); } - + @Override public void setArchived(boolean archived) { - this.archived = archived; + this.archived = archived; } - + @Override public boolean getArchived() { - return archived; + return archived; } private void fireWorkloadStopped() { @@ -187,25 +189,25 @@ public Workload getWorkload() { public void setWorkload(Workload workload) { this.workload = workload; } - + public String[] getOpInfo(){ - return opInfo; + return opInfo; } - + public void setOpInfo(String[] opInfo){ - this.opInfo = opInfo; + this.opInfo = opInfo; } @Override public String[] getAllOperations() { - if(opInfo == null) { + if(opInfo == null) { Set ops = new LinkedHashSet(); for (Stage stage : workload.getWorkflow()) for (Work work : stage) for (Operation op : work) ops.add(op.getType()); setOpInfo(ops.toArray(new String[ops.size()])); - } + } return getOpInfo(); } @@ -269,49 +271,49 @@ public void setReport(Report report) { } public void addListener(WorkloadListener listener) { - if(listeners == null) - return; + if(listeners == null) + return; listeners.add(listener); } - + public HashMap getErrorStatistics() { - return errorStatistics; - } - + return errorStatistics; + } + public void mergeErrorStatistics(){ - for(StageContext stageContext : stageRegistry){ - for(TaskContext taskContext : stageContext.getTaskRegistry()){ - String driverUrl = taskContext.getSchedule().getDriver().getUrl(); - if (! errorStatistics.containsKey(driverUrl)) - errorStatistics.put(driverUrl, new ErrorSummary(taskContext.getErrorStatistics())); - else { - HashMap source = new HashMap(); - source = taskContext.getErrorStatistics(); - HashMap merge = errorStatistics.get(driverUrl).getErrorCodeAndNum(); - for(Map.Entry entry : source.entrySet()){ - if (!merge.containsKey(entry.getKey())){ - merge.put(entry.getKey(), entry.getValue()); - } - else{ - Integer value = merge.get(entry.getKey()) + entry.getValue(); - merge.put(entry.getKey(), value); - } - } - errorStatistics.put(driverUrl, new ErrorSummary(merge)); - } - } - } + for(StageContext stageContext : stageRegistry){ + for(TaskContext taskContext : stageContext.getTaskRegistry()){ + String driverUrl = taskContext.getSchedule().getDriver().getUrl(); + if (! errorStatistics.containsKey(driverUrl)) + errorStatistics.put(driverUrl, new ErrorSummary(taskContext.getErrorStatistics())); + else { + HashMap source = new HashMap(); + source = taskContext.getErrorStatistics(); + HashMap merge = errorStatistics.get(driverUrl).getErrorCodeAndNum(); + for(Map.Entry entry : source.entrySet()){ + if (!merge.containsKey(entry.getKey())){ + merge.put(entry.getKey(), entry.getValue()); + } + else{ + Integer value = merge.get(entry.getKey()) + entry.getValue(); + merge.put(entry.getKey(), value); + } + } + errorStatistics.put(driverUrl, new ErrorSummary(merge)); + } + } + } } public void logErrorStatistics(Logger logger){ - for (Map.Entry driverEntry : errorStatistics.entrySet()){ - for (Map.Entry codeEntry : driverEntry.getValue().getErrorCodeAndNum().entrySet()){ - logger.error(driverEntry.getKey() + " : " + codeEntry.getKey() + " occured " + codeEntry.getValue() ); - } - } + for (Map.Entry driverEntry : errorStatistics.entrySet()){ + for (Map.Entry codeEntry : driverEntry.getValue().getErrorCodeAndNum().entrySet()){ + logger.warn(driverEntry.getKey() + " : " + codeEntry.getKey() + " occured " + codeEntry.getValue() ); + } + } } - @Override + @Override public void disposeRuntime() { for (StageContext stage : stageRegistry) stage.disposeRuntime(); @@ -320,7 +322,7 @@ public void disposeRuntime() { currentStage = null; listeners = null; } - + public DriverRegistry getDriverRegistry() { return driverRegistry; @@ -329,7 +331,7 @@ public DriverRegistry getDriverRegistry() { public void setDriverRegistry(DriverRegistry driverRegistry) { this.driverRegistry = driverRegistry; } - + @Override public DriverInfo[] getDriverInfos() { return driverRegistry.getAllDrivers(); diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/WorkloadListener.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/WorkloadListener.java index 8448f1dc..2d45808a 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/model/WorkloadListener.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/model/WorkloadListener.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.model; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/RAMWorkloadRepository.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/RAMWorkloadRepository.java index 71b106c3..6986da9c 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/RAMWorkloadRepository.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/RAMWorkloadRepository.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.repository; @@ -29,9 +29,9 @@ /** * This class represents one in-memory repository to store all workloads * information. - * + * * @author ywang19, qzheng7 - * + * */ public class RAMWorkloadRepository implements WorkloadRepository, WorkloadListener { @@ -53,7 +53,7 @@ public RAMWorkloadRepository() { WorkloadList workloads = new SimpleWorkloadList(getMaxCapacity()); this.workloads = workloads; } - + @Override public int getMaxCapacity() { int maxCapacity = MAX_WORKLOAD_DEFAULT; @@ -79,11 +79,11 @@ public synchronized void saveWorkload(WorkloadContext workload) { LOGGER.debug("workload {} has been saved in RAM", workload.getId()); LOGGER.debug("{} workloads have been removed from RAM", removed.length); } - + @Override public synchronized void removeWorkload(WorkloadContext workload) { - workloads.remove(workload); - } + workloads.remove(workload); + } @Override public synchronized WorkloadContext getWorkload(String id) { @@ -113,10 +113,10 @@ public synchronized WorkloadContext[] getInactiveWorkloads() { result.add(workload); return result.toArray(new WorkloadContext[result.size()]); } - + @Override public synchronized WorkloadContext[] getArchivedWorkloads() { - List result = new ArrayList(); + List result = new ArrayList(); for (WorkloadContext workload : workloads.values()) if (workload.getArchived()) result.add(workload); diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/SimpleWorkloadList.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/SimpleWorkloadList.java index ccb8aa0c..5c4e8f37 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/SimpleWorkloadList.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/SimpleWorkloadList.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.repository; @@ -51,17 +51,17 @@ public WorkloadContext fetch(String id) { public WorkloadContext[] add(WorkloadContext workload) { toBeRemoved.clear(); // begin transaction list.put(workload.getId(), workload); - capacity = count(); // size of list + capacity = count(); // size of list shrinkListSize(); WorkloadContext[] result = new WorkloadContext[toBeRemoved.size()]; result = toBeRemoved.toArray(result); toBeRemoved.clear(); // end transaction return result; } - + @Override public void remove(WorkloadContext workload) { - list.remove(workload.getId()); + list.remove(workload.getId()); } private void shrinkListSize() { diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/WorkloadList.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/WorkloadList.java index 0feee62e..633fd41c 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/WorkloadList.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/WorkloadList.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.repository; @@ -21,7 +21,8 @@ import com.intel.cosbench.controller.model.WorkloadContext; -interface WorkloadList { +interface WorkloadList + { public int count(); @@ -31,6 +32,6 @@ interface WorkloadList { public WorkloadContext[] add(WorkloadContext workload); - public void remove(WorkloadContext workload); + public void remove(WorkloadContext workload); } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/WorkloadRepository.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/WorkloadRepository.java index d0340ae8..7b31c584 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/WorkloadRepository.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/repository/WorkloadRepository.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.repository; @@ -24,7 +24,7 @@ public interface WorkloadRepository { public int getSize(); public void saveWorkload(WorkloadContext workload); - + public void removeWorkload(WorkloadContext workload); public WorkloadContext getWorkload(String id); @@ -34,9 +34,9 @@ public interface WorkloadRepository { public WorkloadContext[] getActiveWorkloads(); public WorkloadContext[] getInactiveWorkloads(); - + public WorkloadContext[] getArchivedWorkloads(); - + public int getMaxCapacity(); } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/AbstractScheduler.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/AbstractScheduler.java index 72e6085b..793dda2e 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/AbstractScheduler.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/AbstractScheduler.java @@ -1,6 +1,7 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -12,8 +13,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.schedule; @@ -24,9 +25,9 @@ /** * The base class of scheduler. - * + * * @author ywang19, qzheng7 - * + * */ abstract class AbstractScheduler implements WorkScheduler { @@ -39,10 +40,16 @@ public AbstractScheduler() { } protected void init(Stage stage, DriverRegistry registry) { + Boolean useOnlyActiveDrivers = Boolean.parseBoolean( + System.getProperty("cosbench.controller.use_only_active_drivers", "false") + ); for (Work work : stage) works.add(work); - for (DriverContext driver : registry) - drivers.put(driver.getName(), driver); + for (DriverContext driver : registry) { + if (!useOnlyActiveDrivers || driver.getAliveState()) { + drivers.put(driver.getName(), driver); + } + } } protected static SchedulePlan createSchedule(Work work, diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/BalancedScheduler.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/BalancedScheduler.java index b8c71040..89bdc494 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/BalancedScheduler.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/BalancedScheduler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.schedule; @@ -27,9 +27,9 @@ /** * This class encapsulates one balanced scheduler, which tries best to evenly * distribute work to different driver. - * + * * @author ywang19, qzheng7 - * + * */ class BalancedScheduler extends AbstractScheduler { diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/ScheduleException.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/ScheduleException.java index 41968e3a..18ebc302 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/ScheduleException.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/ScheduleException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.schedule; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/Schedulers.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/Schedulers.java index 0f192fe2..7f64a51d 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/Schedulers.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/Schedulers.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.schedule; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/WorkScheduler.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/WorkScheduler.java index 445aa177..82ba1ba6 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/WorkScheduler.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/schedule/WorkScheduler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.schedule; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/COSBControllerService.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/COSBControllerService.java index 8d0944e5..730d1292 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/COSBControllerService.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/COSBControllerService.java @@ -1,389 +1,393 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.controller.service; - -import java.io.File; -import java.io.FileNotFoundException; -import java.io.IOException; -import java.util.*; -import java.util.concurrent.*; -import java.util.concurrent.atomic.AtomicInteger; - -import org.apache.commons.lang.StringUtils; - -import com.intel.cosbench.config.XmlConfig; -import com.intel.cosbench.controller.archiver.*; -import com.intel.cosbench.controller.loader.SimpleWorkloadLoader; -import com.intel.cosbench.controller.model.*; -import com.intel.cosbench.controller.repository.*; -import com.intel.cosbench.log.*; -import com.intel.cosbench.model.*; -import com.intel.cosbench.service.ControllerService; -import com.intel.cosbench.service.WorkloadLoader; - -/** - * This class is the major service for controller. - * - * @author ywang19, qzheng7 - * - */ -class COSBControllerService implements ControllerService, WorkloadListener { - - private static final Logger LOGGER = LogFactory.getSystemLogger(); - - private AtomicInteger count; /* workload id generator */ - - private AtomicInteger order; - - private ControllerContext context; - private Map processors; - private OrderThreadPoolExecutor executor; - private WorkloadArchiver archiver = new SimpleWorkloadArchiver(); - private WorkloadLoader loader = new SimpleWorkloadLoader(); - private WorkloadRepository memRepo = new RAMWorkloadRepository(); - - private boolean loadArch = false; - - public COSBControllerService() { - /* empty */ - } - - public void setContext(ControllerContext context) { - this.context = context; - - // ping drivers and set alive state - Thread pingDriverThread = new Thread( - new PingDriverRunner(context.getDriverInfos())); - pingDriverThread.start(); - } - - public void init() { - if(this.context == null) { - LOGGER.error("Controller Context is not initialized."); - System.exit(-1); - } - - // initialize workload archiver and loader - String archive_dir = context.getArchive_dir(); - archiver = new SimpleWorkloadArchiver(archive_dir); - loader = new SimpleWorkloadLoader(archive_dir); - - count = new AtomicInteger(archiver.getTotalWorkloads()); - order = new AtomicInteger(0); - processors = new HashMap(); - processors = Collections.synchronizedMap(processors); - int concurrency = context.getConcurrency(); - executor = new OrderThreadPoolExecutor(concurrency, concurrency, 0L, - TimeUnit.MILLISECONDS, new PriorityBlockingQueue( - memRepo.getMaxCapacity(), - new OrderFutureComparator())); - - } - - public void loadArchivedWorkload() throws IOException { - List workloadContexts = loader.loadWorkloadRun(); - if (workloadContexts == null) - return; - for (WorkloadInfo workloadContext : workloadContexts) - memRepo.saveWorkload((WorkloadContext) workloadContext); - } - - public void unloadArchivedWorkload() { - for(WorkloadContext workload : memRepo.getArchivedWorkloads()) { - memRepo.removeWorkload(workload); - workload = null; - } - } - - - @Override - public String submit(XmlConfig config) { - LOGGER.debug("[ CT ] - submitting workload ... "); - WorkloadContext workload = createWorkloadContext(config); - WorkloadProcessor processor = createProcessor(workload); - workload.addListener(this); - processors.put(workload.getId(), processor); - memRepo.saveWorkload(workload); - LOGGER.debug("[ CT ] - workload {} submitted", workload.getId()); - return workload.getId(); - } - - @Override - public String resubmit(String id) throws IOException { - XmlConfig config = SimpleWorkloadLoader.getWorkloadConfg(memRepo - .getWorkload(id)); - if (config != null) - return submit(config); - LOGGER.debug( - "[ CT ] - workload {} resubmitted failed, has no workload config", - id); - throw new FileNotFoundException("configuration file for workload " + id); - } - - private WorkloadContext createWorkloadContext(XmlConfig config) { - WorkloadContext context = new WorkloadContext(); - context.setId(generateWorkloadId()); - context.setOrder(generateOrder()); - context.setSubmitDate(new Date()); - context.setConfig(config); - context.setState(WorkloadState.QUEUING); - return context; - } - - public WorkloadLoader getWorkloadLoader() { - return loader; - } - - public boolean getloadArch() { - return loadArch; - } - - public void setloadArch(boolean loadArch) { - this.loadArch = loadArch; - - if(getloadArch()){ - try { - loadArchivedWorkload(); - } catch (IOException e) { - e.printStackTrace(); - } - } else { - unloadArchivedWorkload(); - } - } - - private String generateWorkloadId() { - return "w" + count.incrementAndGet(); - } - - private int generateOrder() { - return order.incrementAndGet(); - } - - private WorkloadProcessor createProcessor(WorkloadContext workload) { - WorkloadProcessor processor = new WorkloadProcessor(); - processor.setControllerContext(context); - processor.setWorkloadContext(workload); - processor.init(); - return processor; - } - - @Override - public void fire(String id) { - if(id == null) { - throw new IllegalStateException("invalid workload id."); - } - final WorkloadProcessor processor = processors.get(id); - if (processor == null) { - throw new IllegalStateException("workload processor is not initialized."); - } - LOGGER.debug("[ CT ] - starting workload {} ...", id); - /* for strong consistency: a lock should be employed here */ - if (processor.getWorkloadContext().getFuture() != null) - throw new IllegalStateException(); - ControllerThread ctrlThrd = new ControllerThread(processor); - Future future = executor.submit(ctrlThrd); - processor.getWorkloadContext().setFuture(future); - yieldExecution(200); // give workload processor a chance - LOGGER.debug("[ CT ] - workload {} started", id); - } - - @Override - public boolean changeOrder(String id, String neighbourWId, boolean up) { - if (StringUtils.isEmpty(neighbourWId)) { - return changeOrder(id, up); - } - if(neighbourWId.equals(String.valueOf(0)))//multiple checked workload id - return false; - int order = processors.get(id).getWorkloadContext().getOrder(); - int neighOrder = processors.get(neighbourWId).getWorkloadContext() - .getOrder(); - if (!up == order > neighOrder ? true : false) - return false; - - if (processors.get(neighbourWId).getWorkloadContext().getState() != WorkloadState.QUEUING) { - LOGGER.error( - "[ CT ] - workload {} order failed cause it's highest order...", - id); - return false; - } - List orders = new ArrayList(); - Map orderWorkloadMap = new HashMap(); - for (WorkloadContext workload : getActiveWorkloads()) { - if ((workload.getOrder() >= order && workload.getOrder() <= neighOrder) - || (workload.getOrder() <= order && workload - .getOrder() >= neighOrder)) { - orders.add(workload.getOrder()); - orderWorkloadMap.put(String.valueOf(workload.getOrder()), - workload.getId()); - } - } - Integer[] orderArray = orders.toArray(new Integer[orders.size()]); - Arrays.sort(orderArray); - - if (up) { - for (int i = orderArray.length - 2; i >= 0; i--) { - processors.get(orderWorkloadMap.get(String.valueOf(orderArray[i]))) - .getWorkloadContext().setOrder(orderArray[i + 1]); - } - } else { - for (int i = 1; i orders = new ArrayList(); - for(WorkloadContext workload:getActiveWorkloads()){ - orders.add(workload.getOrder()); - } - Integer[] orderArray = orders.toArray(new Integer[orders.size()]); - Arrays.sort(orderArray); - if (up) { - for (int i = orderArray.length - 1; i >= 0; i--) { - if (orderArray[i] < order) { - neighbourOrder = orderArray[i]; - break; - } - } - }else{ - for (int i = 0; i < orderArray.length; i++) { - if (orderArray[i] > order) { - neighbourOrder = orderArray[i]; - break; - } - } - } - String neighbourWId = String.valueOf(0); - for(WorkloadContext workload:getActiveWorkloads()){ - if (workload.getOrder() == neighbourOrder) { - neighbourWId = workload.getId(); - } - } - if (neighbourWId.equals(String.valueOf(0)))//can't find neighbour workload - return false; - if (processors.get(neighbourWId).getWorkloadContext().getState() != WorkloadState.QUEUING) { - LOGGER.debug( - "[ CT ] - workload {} order failed cause it's highest order...", - id); - return false; - } - if (!processors.get(id).getWorkloadContext().getFuture().cancel(true) - || !processors.get(neighbourWId).getWorkloadContext() - .getFuture().cancel(true)) { - LOGGER.error( - "[ CT ] - change workload {} {} order failed cause can't remove workload...", - id, neighbourWId); - return false; - } - processors.get(id).getWorkloadContext().setFuture(null); - processors.get(neighbourWId).getWorkloadContext().setFuture(null); - processors.get(id).getWorkloadContext().setOrder(neighbourOrder); - processors.get(neighbourWId).getWorkloadContext().setOrder(order); - fire(id); - fire(neighbourWId); - return true; - } - - @Override - public void cancel(String id) { - WorkloadProcessor processor = processors.get(id); - if (processor == null) - return; // already stopped - LOGGER.debug("[ CT ] - canceling workload{} ...", id); - processor.cancel(); - yieldExecution(500); // give workload processor a chance - LOGGER.debug("[ CT ] - workload {} cancelled", id); - } - - private static void yieldExecution(int time) { - try { - Thread.sleep(time); - } catch (InterruptedException e) { - LOGGER.warn("get interrupted when performing yield"); - Thread.currentThread().interrupt(); // re-interrupt - } - } - - @Override - public ControllerInfo getControllerInfo() { - return context; - } - - @Override - public WorkloadContext getWorkloadInfo(String id) { - return memRepo.getWorkload(id); - } - - @Override - public WorkloadContext[] getActiveWorkloads() { - return memRepo.getActiveWorkloads(); - } - - @Override - public WorkloadContext[] getHistoryWorkloads() { - return memRepo.getInactiveWorkloads(); - } - - @Override - public WorkloadInfo[] getArchivedWorkloads() { - return memRepo.getArchivedWorkloads(); - } - - @Override - public void workloadStarted(WorkloadContext workload) { - /* empty */ - } - - @Override - public void workloadStopped(WorkloadContext workload) { - String id = workload.getId(); - WorkloadProcessor processor = processors.remove(id); - processor.dispose(); - archiver.archive(workload); - LOGGER.debug("processor for workload {} has been detached", id); - } - - @Override - public File getWorkloadLog(WorkloadInfo info) { - return archiver.getWorkloadLog(info); - } - - @Override - public File getWorkloadConfig(WorkloadInfo info) { - return archiver.getWorkloadConfig(info); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.controller.service; + +import java.io.File; +import java.io.FileNotFoundException; +import java.io.IOException; +import java.util.*; +import java.util.concurrent.*; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.commons.lang.StringUtils; + +import com.intel.cosbench.config.XmlConfig; +import com.intel.cosbench.controller.archiver.*; +import com.intel.cosbench.controller.loader.SimpleWorkloadLoader; +import com.intel.cosbench.controller.model.*; +import com.intel.cosbench.controller.repository.*; +import com.intel.cosbench.log.*; +import com.intel.cosbench.model.*; +import com.intel.cosbench.service.ControllerService; +import com.intel.cosbench.service.WorkloadLoader; + +/** + * This class is the major service for controller. + * + * @author ywang19, qzheng7 + * + */ +class COSBControllerService implements ControllerService, WorkloadListener { + + private static final Logger LOGGER = LogFactory.getSystemLogger(); + + private AtomicInteger count; /* workload id generator */ + + private AtomicInteger order; + + private ControllerContext context; + private Map processors; + private OrderThreadPoolExecutor executor; + private WorkloadArchiver archiver = new SimpleWorkloadArchiver(); + private WorkloadLoader loader = new SimpleWorkloadLoader(); + private WorkloadRepository memRepo = new RAMWorkloadRepository(); + + private boolean loadArch = false; + + public COSBControllerService() { + /* empty */ + } + + public void setContext(ControllerContext context) { + this.context = context; + + // ping drivers and set alive state + Thread pingDriverThread = new Thread( + new PingDriverRunner(context.getDriverInfos())); + pingDriverThread.start(); + } + + public void init() { + if(this.context == null) { + LOGGER.error("Controller Context is not initialized."); + System.exit(-1); + } + + // initialize workload archiver and loader + String archive_dir = context.getArchive_dir(); + archiver = new SimpleWorkloadArchiver(archive_dir); + loader = new SimpleWorkloadLoader(archive_dir); + + count = new AtomicInteger(archiver.getTotalWorkloads()); + order = new AtomicInteger(0); + processors = new HashMap(); + processors = Collections.synchronizedMap(processors); + int concurrency = context.getConcurrency(); + executor = new OrderThreadPoolExecutor(concurrency, concurrency, 0L, + TimeUnit.MILLISECONDS, new PriorityBlockingQueue( + memRepo.getMaxCapacity(), + new OrderFutureComparator())); + + } + + public void loadArchivedWorkload() throws IOException { + List workloadContexts = loader.loadWorkloadRun(); + if (workloadContexts == null) + return; + for (WorkloadInfo workloadContext : workloadContexts) + memRepo.saveWorkload((WorkloadContext) workloadContext); + } + + public void unloadArchivedWorkload() { + for(WorkloadContext workload : memRepo.getArchivedWorkloads()) { + memRepo.removeWorkload(workload); + workload = null; + } + } + + + @Override + public synchronized String submit(XmlConfig config) { + LOGGER.debug("[ CT ] - submitting workload ... "); + WorkloadContext workload = createWorkloadContext(config); + WorkloadProcessor processor = createProcessor(workload); + workload.addListener(this); + processors.put(workload.getId(), processor); + memRepo.saveWorkload(workload); + LOGGER.debug("[ CT ] - workload {} submitted", workload.getId()); + return workload.getId(); + } + + @Override + public String resubmit(String id) throws IOException { + XmlConfig config = SimpleWorkloadLoader.getWorkloadConfg(memRepo + .getWorkload(id)); + if (config != null) + return submit(config); + LOGGER.debug( + "[ CT ] - workload {} resubmitted failed, has no workload config", + id); + throw new FileNotFoundException("configuration file for workload " + id); + } + + private WorkloadContext createWorkloadContext(XmlConfig config) { + WorkloadContext context = new WorkloadContext(); + context.setId(generateWorkloadId()); + context.setOrder(generateOrder()); + context.setSubmitDate(new Date()); + context.setConfig(config); + context.setState(WorkloadState.QUEUING); + return context; + } + + public WorkloadLoader getWorkloadLoader() { + return loader; + } + + public boolean getloadArch() { + return loadArch; + } + + public void setloadArch(boolean loadArch) { + this.loadArch = loadArch; + + if(getloadArch()){ + try { + loadArchivedWorkload(); + } catch (IOException e) { + e.printStackTrace(); + } + } else { + unloadArchivedWorkload(); + } + } + + private String generateWorkloadId() { + return "w" + count.incrementAndGet(); + } + + private int generateOrder() { + return order.incrementAndGet(); + } + + private WorkloadProcessor createProcessor(WorkloadContext workload) { + WorkloadProcessor processor = new WorkloadProcessor(); + processor.setControllerContext(context); + processor.setWorkloadContext(workload); + processor.init(); + return processor; + } + + @Override + public void fire(String id) { + if(id == null) { + throw new IllegalStateException("invalid workload id."); + } + final WorkloadProcessor processor = processors.get(id); + if (processor == null) { + throw new IllegalStateException("workload processor is not initialized."); + } + LOGGER.debug("[ CT ] - starting workload {} ...", id); + /* for strong consistency: a lock should be employed here */ + if (processor.getWorkloadContext().getFuture() != null) + throw new IllegalStateException(); + ControllerThread ctrlThrd = new ControllerThread(processor); + + Future future = null; + synchronized(processor) { + future = executor.submit(ctrlThrd); + processor.getWorkloadContext().setFuture(future); + } + LOGGER.debug("[ CT ] - workload {} started", id); + yieldExecution(200); // give workload processor a chance + } + + @Override + public boolean changeOrder(String id, String neighbourWId, boolean up) { + if (StringUtils.isEmpty(neighbourWId)) { + return changeOrder(id, up); + } + if(neighbourWId.equals(String.valueOf(0)))//multiple checked workload id + return false; + int order = processors.get(id).getWorkloadContext().getOrder(); + int neighOrder = processors.get(neighbourWId).getWorkloadContext() + .getOrder(); + if (!up == order > neighOrder ? true : false) + return false; + + if (processors.get(neighbourWId).getWorkloadContext().getState() != WorkloadState.QUEUING) { + LOGGER.error( + "[ CT ] - workload {} order failed cause it's highest order...", + id); + return false; + } + List orders = new ArrayList(); + Map orderWorkloadMap = new HashMap(); + for (WorkloadContext workload : getActiveWorkloads()) { + if ((workload.getOrder() >= order && workload.getOrder() <= neighOrder) + || (workload.getOrder() <= order && workload + .getOrder() >= neighOrder)) { + orders.add(workload.getOrder()); + orderWorkloadMap.put(String.valueOf(workload.getOrder()), + workload.getId()); + } + } + Integer[] orderArray = orders.toArray(new Integer[orders.size()]); + Arrays.sort(orderArray); + + if (up) { + for (int i = orderArray.length - 2; i >= 0; i--) { + processors.get(orderWorkloadMap.get(String.valueOf(orderArray[i]))) + .getWorkloadContext().setOrder(orderArray[i + 1]); + } + } else { + for (int i = 1; i orders = new ArrayList(); + for(WorkloadContext workload:getActiveWorkloads()){ + orders.add(workload.getOrder()); + } + Integer[] orderArray = orders.toArray(new Integer[orders.size()]); + Arrays.sort(orderArray); + if (up) { + for (int i = orderArray.length - 1; i >= 0; i--) { + if (orderArray[i] < order) { + neighbourOrder = orderArray[i]; + break; + } + } + }else{ + for (int i = 0; i < orderArray.length; i++) { + if (orderArray[i] > order) { + neighbourOrder = orderArray[i]; + break; + } + } + } + String neighbourWId = String.valueOf(0); + for(WorkloadContext workload:getActiveWorkloads()){ + if (workload.getOrder() == neighbourOrder) { + neighbourWId = workload.getId(); + } + } + if (neighbourWId.equals(String.valueOf(0)))//can't find neighbour workload + return false; + if (processors.get(neighbourWId).getWorkloadContext().getState() != WorkloadState.QUEUING) { + LOGGER.debug( + "[ CT ] - workload {} order failed cause it's highest order...", + id); + return false; + } + if (!processors.get(id).getWorkloadContext().getFuture().cancel(true) + || !processors.get(neighbourWId).getWorkloadContext() + .getFuture().cancel(true)) { + LOGGER.error( + "[ CT ] - change workload {} {} order failed cause can't remove workload...", + id, neighbourWId); + return false; + } + processors.get(id).getWorkloadContext().setFuture(null); + processors.get(neighbourWId).getWorkloadContext().setFuture(null); + processors.get(id).getWorkloadContext().setOrder(neighbourOrder); + processors.get(neighbourWId).getWorkloadContext().setOrder(order); + fire(id); + fire(neighbourWId); + return true; + } + + @Override + public void cancel(String id) { + WorkloadProcessor processor = processors.get(id); + if (processor == null) + return; // already stopped + LOGGER.debug("[ CT ] - canceling workload{} ...", id); + processor.cancel(); + yieldExecution(500); // give workload processor a chance + LOGGER.debug("[ CT ] - workload {} cancelled", id); + } + + private static void yieldExecution(int time) { + try { + Thread.sleep(time); + } catch (InterruptedException e) { + LOGGER.warn("get interrupted when performing yield"); + Thread.currentThread().interrupt(); // re-interrupt + } + } + + @Override + public ControllerInfo getControllerInfo() { + return context; + } + + @Override + public WorkloadContext getWorkloadInfo(String id) { + return memRepo.getWorkload(id); + } + + @Override + public WorkloadContext[] getActiveWorkloads() { + return memRepo.getActiveWorkloads(); + } + + @Override + public WorkloadContext[] getHistoryWorkloads() { + return memRepo.getInactiveWorkloads(); + } + + @Override + public WorkloadInfo[] getArchivedWorkloads() { + return memRepo.getArchivedWorkloads(); + } + + @Override + public void workloadStarted(WorkloadContext workload) { + /* empty */ + } + + @Override + public void workloadStopped(WorkloadContext workload) { + String id = workload.getId(); + WorkloadProcessor processor = processors.remove(id); + processor.dispose(); + archiver.archive(workload); + LOGGER.debug("processor for workload {} has been detached", id); + } + + @Override + public File getWorkloadLog(WorkloadInfo info) { + return archiver.getWorkloadLog(info); + } + + @Override + public File getWorkloadConfig(WorkloadInfo info) { + return archiver.getWorkloadConfig(info); + } + +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/COSBControllerServiceFactory.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/COSBControllerServiceFactory.java index 06c320bd..5334033f 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/COSBControllerServiceFactory.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/COSBControllerServiceFactory.java @@ -1,132 +1,162 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.controller.service; - -import static org.apache.commons.lang.SystemUtils.IS_OS_WINDOWS; - -import java.io.File; - -import com.intel.cosbench.config.castor.CastorConfigTools; -import com.intel.cosbench.controller.model.*; -import com.intel.cosbench.service.*; - -public class COSBControllerServiceFactory extends AbstractServiceFactory - implements ControllerServiceFactory { - - private static final String SERVICE_NAME = "controller"; - - private static final String CFG_FILE_KEY = "cosbench.controller.config"; - - private static final String UNIX_DEFAULT_CFG_FILE = "/etc/cosbench/controller.conf"; - - private static final String WIN_DEFAULT_CFG_FILE = "C:\\controller.conf"; - - public COSBControllerServiceFactory() { - /* loading workload XML mappings */ - CastorConfigTools.getWorkloadResolver(); -// /* creating workload archive directory */ -// new SimpleWorkloadArchiver(); - } - - @Override - protected String getConfigFile() { - String configFile; - if ((configFile = System.getProperty(CFG_FILE_KEY)) != null) - return configFile; - if (new File("controller.conf").exists()) - return "controller.conf"; - if (new File("conf/controller.conf").exists()) - return "conf/controller.conf"; - return IS_OS_WINDOWS ? WIN_DEFAULT_CFG_FILE : UNIX_DEFAULT_CFG_FILE; - } - - @Override - protected String getServiceName() { - return SERVICE_NAME; - } - - @Override - public ControllerService getControllerService() { - COSBControllerService service = new COSBControllerService(); - ControllerContext context = getControllerContext(); - service.setContext(context); - service.init(); - return service; - } - - private ControllerContext getControllerContext() { - ControllerContext context = new ControllerContext(); - context.setName(loadControllerName()); - context.setUrl(loadControllerUrl()); - context.setArchive_dir(loadArchiveDir()); - context.setConcurrency(loadConcurrency()); - context.setDriverRegistry(getDriverRegistry()); - return context; - } - - protected String loadLogLevel() { - return config.get("controller.log_level", "INFO"); - } - - protected String loadLogFile() { - return config.get("controller.log_file", "log/system.log"); - } - - private String loadArchiveDir() { - return config.get("controller.archive_dir", "archive"); - } - - private String loadControllerName() { - return config.get("controller.name", "N/A"); - } - - private String loadControllerUrl() { - return config.get("controller.url", "N/A"); - } - - private int loadConcurrency() { - return config.getInt("controller.concurrency", 1); - } - - private DriverRegistry getDriverRegistry() { - DriverRegistry registry = new DriverRegistry(); - int drivers = config.getInt("controller.drivers"); - for (int i = 1; i <= drivers; i++) { - DriverContext context = getDriverContext(i); - registry.addDriver(context); - } - return registry; - } - - private DriverContext getDriverContext(int index) { - DriverContext context = new DriverContext(); - context.setName(loadDriverName(index)); - context.setUrl(loadDriverUrl(index)); - context.setAliveState(false); - return context; - } - - private String loadDriverName(int index) { - return config.get("driver" + index + ".name"); - } - - private String loadDriverUrl(int index) { - return config.get("driver" + index + ".url"); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.controller.service; + +import static org.apache.commons.lang.SystemUtils.IS_OS_WINDOWS; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileReader; +import java.io.IOException; +import java.io.Reader; +import java.nio.channels.ReadableByteChannel; + +import com.intel.cosbench.config.castor.CastorConfigTools; +import com.intel.cosbench.controller.model.*; +import com.intel.cosbench.service.*; + +public class COSBControllerServiceFactory extends AbstractServiceFactory + implements ControllerServiceFactory { + + private static final String SERVICE_NAME = "controller"; + + private static final String CFG_FILE_KEY = "cosbench.controller.config"; + + private static final String UNIX_DEFAULT_CFG_FILE = "/etc/cosbench/controller.conf"; + + private static final String WIN_DEFAULT_CFG_FILE = "C:\\controller.conf"; + + public COSBControllerServiceFactory() { + /* loading workload XML mappings */ + CastorConfigTools.getWorkloadResolver(); +// /* creating workload archive directory */ +// new SimpleWorkloadArchiver(); + } + + @Override + protected String getConfigFile() { + String configFile; + if ((configFile = System.getProperty(CFG_FILE_KEY)) != null) + return configFile; + if (new File("controller.conf").exists()) + return "controller.conf"; + if (new File("conf/controller.conf").exists()) + return "conf/controller.conf"; + return IS_OS_WINDOWS ? WIN_DEFAULT_CFG_FILE : UNIX_DEFAULT_CFG_FILE; + } + + @Override + protected String getServiceName() { + return SERVICE_NAME; + } + + @Override + public ControllerService getControllerService() { + COSBControllerService service = new COSBControllerService(); + ControllerContext context = getControllerContext(); + service.setContext(context); + service.init(); + return service; + } + + private ControllerContext getControllerContext() { + ControllerContext context = new ControllerContext(); + context.setName(loadControllerName()); + context.setUrl(loadControllerUrl()); + context.setArchive_dir(loadArchiveDir()); + context.setConcurrency(loadConcurrency()); + context.setDriverRegistry(getDriverRegistry()); + context.setVersion(getVersion()); + return context; + } + + private String getVersion() { + // TODO Auto-generated method stub + String str = getName("VERSION"); + String str2 = getName("BUILD.no"); + return str+"."+str2; + } + + private String getName(String fileName){ + String str = null ; + File myFile=new File(fileName); + if(!myFile.exists()){ + System.err.println("Can't Find " + fileName); + } + try { + BufferedReader in = new BufferedReader(new FileReader(myFile)); + str = in.readLine(); + in.close(); + } + catch (IOException e) { + e.getStackTrace(); + } + return str; + } + + protected String loadLogLevel() { + return config.get("controller.log_level", "INFO"); + } + + protected String loadLogFile() { + return config.get("controller.log_file", "log/system.log"); + } + + private String loadArchiveDir() { + return config.get("controller.archive_dir", "archive"); + } + + private String loadControllerName() { + return config.get("controller.name", "N/A"); + } + + private String loadControllerUrl() { + return config.get("controller.url", "N/A"); + } + + private int loadConcurrency() { + return config.getInt("controller.concurrency", 1); + } + + private DriverRegistry getDriverRegistry() { + DriverRegistry registry = new DriverRegistry(); + int drivers = config.getInt("controller.drivers"); + for (int i = 1; i <= drivers; i++) { + DriverContext context = getDriverContext(i); + registry.addDriver(context); + } + return registry; + } + + private DriverContext getDriverContext(int index) { + DriverContext context = new DriverContext(); + context.setName(loadDriverName(index)); + context.setUrl(loadDriverUrl(index)); + context.setAliveState(false); + return context; + } + + private String loadDriverName(int index) { + return config.get("driver" + index + ".name"); + } + + private String loadDriverUrl(int index) { + return config.get("driver" + index + ".url"); + } + +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/ControllerThread.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/ControllerThread.java index 91f62f40..f3bfcf06 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/ControllerThread.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/ControllerThread.java @@ -1,25 +1,43 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.service; class ControllerThread implements Runnable { - private int order; - private WorkloadProcessor processor; - - public ControllerThread(WorkloadProcessor processor) { - this.processor = processor; - setOrder(processor.getWorkloadContext().getOrder()); - } - - @Override - public void run() { - processor.process(); - processor.getWorkloadContext().setFuture(null); - } - - public int getOrder() { - return this.order; - } - - public void setOrder(int order) { - this.order = order; - } -} \ No newline at end of file + private int order; + private WorkloadProcessor processor; + + public ControllerThread(WorkloadProcessor processor) { + this.processor = processor; + setOrder(processor.getWorkloadContext().getOrder()); + } + + @Override + public void run() { + processor.process(); + processor.getWorkloadContext().setFuture(null); + } + + public int getOrder() { + return this.order; + } + + public void setOrder(int order) { + this.order = order; + } +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderFuture.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderFuture.java index e19854a4..442e9a3b 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderFuture.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderFuture.java @@ -1,17 +1,35 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.service; import java.util.concurrent.FutureTask; class OrderFuture extends FutureTask { - private int order; - - public OrderFuture(Runnable runnable, T value, int order) { - super(runnable, value); - this.order = order; - } + private int order; + + public OrderFuture(Runnable runnable, T value, int order) { + super(runnable, value); + this.order = order; + } - public int getOrder() { - return order; - } + public int getOrder() { + return order; + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderFutureComparator.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderFutureComparator.java index 79894bfd..b217df10 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderFutureComparator.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderFutureComparator.java @@ -1,20 +1,38 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.service; import java.util.Comparator; class OrderFutureComparator implements Comparator { - public int compare(Runnable o1, Runnable o2) { - if (o1 == null && o2 == null) - return 0; - else if (o1 == null) - return -1; - else if (o2 == null) - return 1; - else { - int p1 = ((OrderFuture) o1).getOrder(); - int p2 = ((OrderFuture) o2).getOrder(); - - return p1 > p2 ? 1 : (p1 == p2 ? 0 : -1); - } - } + public int compare(Runnable o1, Runnable o2) { + if (o1 == null && o2 == null) + return 0; + else if (o1 == null) + return -1; + else if (o2 == null) + return 1; + else { + int p1 = ((OrderFuture) o1).getOrder(); + int p2 = ((OrderFuture) o2).getOrder(); + + return p1 > p2 ? 1 : (p1 == p2 ? 0 : -1); + } + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderThreadPoolExecutor.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderThreadPoolExecutor.java index ab8670c6..6c3428a6 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderThreadPoolExecutor.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/OrderThreadPoolExecutor.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.service; import java.util.concurrent.BlockingQueue; import java.util.concurrent.Future; @@ -7,21 +25,21 @@ public class OrderThreadPoolExecutor extends ThreadPoolExecutor { - public OrderThreadPoolExecutor(int corePoolSize, int maximumPoolSize, - long keepAliveTime, TimeUnit unit, BlockingQueue workQueue) { - super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue); - } - - protected RunnableFuture newTaskFor(Runnable runnable, T value) { - return new OrderFuture(runnable, null, - ((ControllerThread) runnable).getOrder()); - } - - public Future submit(Runnable task) { - if (task == null) - throw new NullPointerException(); - RunnableFuture ftask = newTaskFor(task, null); - execute(ftask); - return ftask; - } + public OrderThreadPoolExecutor(int corePoolSize, int maximumPoolSize, + long keepAliveTime, TimeUnit unit, BlockingQueue workQueue) { + super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue); + } + + protected RunnableFuture newTaskFor(Runnable runnable, T value) { + return new OrderFuture(runnable, null, + ((ControllerThread) runnable).getOrder()); + } + + public Future submit(Runnable task) { + if (task == null) + throw new NullPointerException(); + RunnableFuture ftask = newTaskFor(task, null); + execute(ftask); + return ftask; + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/PingDriverRunner.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/PingDriverRunner.java index 30b9bbee..7166d8c3 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/PingDriverRunner.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/PingDriverRunner.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,66 +12,129 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.service; -import java.net.InetAddress; +import java.io.IOException; import java.net.InetSocketAddress; import java.net.Socket; +import com.intel.cosbench.log.LogFactory; +import com.intel.cosbench.log.Logger; import com.intel.cosbench.model.DriverInfo; public class PingDriverRunner implements Runnable{ - private int interval = 5000; - private DriverInfo[] driverInfos; - - PingDriverRunner(DriverInfo[] driverInfos){ - this.driverInfos = driverInfos; - } - - @Override - public void run() { - while (true) { - pingDrivers(driverInfos); - try { - Thread.sleep(interval); - } catch (InterruptedException ignore) { - } - } - } - - private void pingDrivers(DriverInfo[] driverInfos) { - for (DriverInfo driver : driverInfos) { - boolean isAlive = false; - - String ipAddress = getIpAddres(driver.getUrl()); - try { - if (!ipAddress.isEmpty()) { - try{ - Socket socket = new Socket(); - InetSocketAddress reAddress = new InetSocketAddress(ipAddress, 18088); - InetSocketAddress locAddress = new InetSocketAddress("127.0.0.1", 0); - socket.bind(locAddress); - socket.connect(reAddress,3000); - isAlive = true; - }catch(Exception e){ - isAlive = false; - } - } - }finally{ - driver.setAliveState(isAlive); - } - } - } - - private String getIpAddres(String url) { - int start = url.indexOf('/') + 2; - int end = url.lastIndexOf(':'); - return end > start ? url.substring(start, end) : null; - } - + protected static final Logger LOGGER = LogFactory.getSystemLogger(); + + private static final int base_interval = 20; // heartbeat interval in seconds. + private static final int failure_tolerance = 3; // how many failures in a row is tolerable. + private static final int ext_interval = 3*base_interval; // extended interval for the case failures is over tolerance. + private static final int delay = 3; // timeout for socket wait in seconds + private DriverInfo[] driverInfos; + private int driver_count = 0; + private int[] failures; + private boolean[] all_alive; + + PingDriverRunner(DriverInfo[] driverInfos){ + this.driverInfos = driverInfos; + this.driver_count = driverInfos.length; + this.failures = new int[this.driver_count]; + this.all_alive = new boolean[this.driver_count]; + } + + public boolean isAlive() { + for(boolean status : all_alive) { + if(!status) + return false; + } + + return true; + } + + public void checkFaultyDrivers() { + for(int i=0; i= failure_tolerance) { + all_alive[i] = false; + LOGGER.warn("The driver " + driver.getName() + " at " + driver.getUrl() + " hits failure tolerance, will extend the heartbeat interval to " + ext_interval + " seconds"); + } + }finally{ + driver.setAliveState(all_alive[i]); + if (socket != null) { + try { + socket.close(); + }catch (IOException ignore) { + } + } + } + + } + } + + private String getIpAddres(String url) { + int start = url.indexOf('/') + 2; + int end = url.lastIndexOf(':'); + return end > start ? url.substring(start, end) : null; + } + + private Integer getDriverPort(String url) { + int start = url.lastIndexOf(":")+1; + int end = url.lastIndexOf("/"); + return end > start? Integer.valueOf(url.substring(start, end)):null; + } + } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageCallable.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageCallable.java index c73f7d44..1ff95843 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageCallable.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageCallable.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.service; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageChecker.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageChecker.java index 239a6d7a..59ab39f2 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageChecker.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageChecker.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.service; @@ -25,9 +25,9 @@ /** * This class encapsulates operations to check stage status. - * + * * @author ywang19, qzheng7 - * + * */ public class StageChecker implements StageCallable { diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageException.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageException.java index ac6dc8df..1a2ffac0 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageException.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.service; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageRunner.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageRunner.java index 67fcffe7..d021efa3 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageRunner.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/StageRunner.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.service; @@ -34,9 +34,9 @@ /** * This class encapsulates one control class to run stages. - * + * * @author ywang19, qzheng7 - * + * */ class StageRunner implements StageCallable { @@ -138,50 +138,50 @@ private void runStage() { if (Thread.interrupted()) throw new CancelledException(); closeTasks(); - - TaskRegistry tasks = stageContext.getTaskRegistry(); - for (TaskContext task : tasks) { - if (task.getState().equals(TaskState.FAILED)) { - stageContext.setState(FAILED); - return; - } - } - - if (!reachAFRGoal()) { - stageContext.setState(FAILED); - return; - } + + TaskRegistry tasks = stageContext.getTaskRegistry(); + for (TaskContext task : tasks) { + if (task.getState().equals(TaskState.FAILED)) { + stageContext.setState(FAILED); + return; + } + } + + if (!reachAFRGoal()) { + stageContext.setState(FAILED); + return; + } stageContext.setState(COMPLETED); } - + private boolean reachAFRGoal() { - String id = stageContext.getId(); - boolean bool = true; - stageContext.setReport(stageContext.mergeReport()); - for (Work work : stageContext.getStage().getWorks()) { - List operationIDs = work.getOperationIDs(); - long sumSampleCount = 0; - long sumTotalSampleCount = 0; - for (Metrics metric : stageContext.getReport().getAllMetrics()) { - if (operationIDs.contains(metric.getOpId())) { - sumSampleCount += - metric.getSampleCount() > 0 ? metric.getSampleCount() : 0; - sumTotalSampleCount += - metric.getTotalSampleCount() > 0 ? metric.getTotalSampleCount() : 0; - } - } - LOGGER.info("acceptable failure ratio of work {} = {}", id+"-"+work.getName(), (double)work.getAfr() / 1000000); - LOGGER.info("real failure ratio of work {} = {}", id+"-"+work.getName(), - sumTotalSampleCount > 0 ? (double)(sumTotalSampleCount - sumSampleCount) / sumTotalSampleCount : "N/A"); - if ((sumTotalSampleCount - sumSampleCount) > sumTotalSampleCount * work.getAfr() / 1000000) { - LOGGER.info("fail to reach the goal of acceptable failure ratio in stage {} - work {}", id, work.getName()); - bool = false; - continue; - } - LOGGER.info("successfully reach the goal of acceptable failure ratio in stage {} - work {}", id, work.getName()); - } - return bool; - } + String id = stageContext.getId(); + boolean bool = true; + stageContext.setReport(stageContext.mergeReport()); + for (Work work : stageContext.getStage().getWorks()) { + List operationIDs = work.getOperationIDs(); + long sumSampleCount = 0; + long sumTotalSampleCount = 0; + for (Metrics metric : stageContext.getReport().getAllMetrics()) { + if (operationIDs.contains(metric.getOpId())) { + sumSampleCount += + metric.getSampleCount() > 0 ? metric.getSampleCount() : 0; + sumTotalSampleCount += + metric.getTotalSampleCount() > 0 ? metric.getTotalSampleCount() : 0; + } + } + LOGGER.info("acceptable failure ratio of work {} = {}", id+"-"+work.getName(), (double)work.getAfr() / 1000000); + LOGGER.info("real failure ratio of work {} = {}", id+"-"+work.getName(), + sumTotalSampleCount > 0 ? (double)(sumTotalSampleCount - sumSampleCount) / sumTotalSampleCount : "N/A"); + if ((sumTotalSampleCount - sumSampleCount) > sumTotalSampleCount * work.getAfr() / 1000000) { + LOGGER.info("fail to reach the goal of acceptable failure ratio in stage {} - work {}", id, work.getName()); + bool = false; + continue; + } + LOGGER.info("successfully reach the goal of acceptable failure ratio in stage {} - work {}", id, work.getName()); + } + return bool; + } private void bootTasks() { String id = stageContext.getId(); diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/TriggerRunner.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/TriggerRunner.java index 6be48c24..827bdb43 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/TriggerRunner.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/TriggerRunner.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.service; import java.util.List; @@ -11,43 +29,43 @@ import com.intel.cosbench.log.Logger; public class TriggerRunner { - DriverRegistry registry; - private ExecutorService executor; + DriverRegistry registry; + private ExecutorService executor; private static final Logger LOGGER = LogFactory.getSystemLogger(); - - public TriggerRunner(DriverRegistry registry) { - this.registry = registry; - createExecutor(); - } - - public void runTrigger(boolean option, String trigger, String wsId) { - List tasklets = Tasklets.newTriggers(trigger, registry, option, wsId); - executeTasklets(tasklets, option); - dispose(); - } + + public TriggerRunner(DriverRegistry registry) { + this.registry = registry; + createExecutor(); + } + + public void runTrigger(boolean option, String trigger, String wsId) { + List tasklets = Tasklets.newTriggers(trigger, registry, option, wsId); + executeTasklets(tasklets, option); + dispose(); + } private void createExecutor() { int taskCount = registry.getSize(); executor = Executors.newFixedThreadPool(taskCount); } - + public void dispose() { if (executor != null) executor.shutdown(); executor = null; } - + private void executeTasklets(List tasklets, boolean option) { - int num = tasklets.size(); - LOGGER.debug("begin to execute {}-trigger tasklets, {} in total", - option ? "enable" : "kill", num); - try { - executor.invokeAll(tasklets); - } catch (InterruptedException e) { - LOGGER.debug("{}-trigger tasklets have interrupted", - option ? "enable" : "kill", num); - return; //no return is ok? - } - LOGGER.debug("all {} trigger tasklets have finished execution", num); + int num = tasklets.size(); + LOGGER.debug("begin to execute {}-trigger tasklets, {} in total", + option ? "enable" : "kill", num); + try { + executor.invokeAll(tasklets); + } catch (InterruptedException e) { + LOGGER.debug("{}-trigger tasklets have interrupted", + option ? "enable" : "kill", num); + return; //no return is ok? + } + LOGGER.debug("all {} trigger tasklets have finished execution", num); } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/WorkloadException.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/WorkloadException.java index 62fafa1d..0ad9a820 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/WorkloadException.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/WorkloadException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.service; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/WorkloadProcessor.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/WorkloadProcessor.java index 99519a55..8a3c3ed4 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/service/WorkloadProcessor.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/service/WorkloadProcessor.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.service; @@ -23,6 +23,8 @@ import java.util.*; import java.util.concurrent.*; +import org.apache.commons.lang.StringUtils; + import com.intel.cosbench.config.*; import com.intel.cosbench.config.castor.CastorConfigTools; import com.intel.cosbench.controller.model.*; @@ -33,9 +35,9 @@ /** * This class encapsulates workload processing logic. - * + * * @author ywang19, qzheng7 - * + * */ class WorkloadProcessor { @@ -72,12 +74,12 @@ public void init() { } public void dispose() { - if (executor != null) - executor.shutdown(); - executor = null; - } + if (executor != null) + executor.shutdown(); + executor = null; + } - private void resolveWorklaod() { + private void resolveWorklaod() { XmlConfig config = workloadContext.getConfig(); WorkloadResolver resolver = CastorConfigTools.getWorkloadResolver(); workloadContext.setWorkload(resolver.toWorkload(config)); @@ -94,20 +96,21 @@ private void createStages() { } private static StageContext createStageContext(String id, Stage stage) { - initStageOpId(stage); + initStageOpId(stage); StageContext context = new StageContext(); context.setId(id); context.setStage(stage); context.setState(StageState.WAITING); return context; } - + private static void initStageOpId(Stage stage) { - int index = 0; - for (Work work : stage.getWorks()) { - for (Operation op : work.getOperations()) - op.setId("op" + String.valueOf(++index)); - } + int index = 0; + for (Work work : stage.getWorks()) { + for (Operation op : work.getOperations()) + if("none".equals(op.getId())) + op.setId("op" + String.valueOf(++index)); + } } private void createExecutor() { @@ -131,10 +134,10 @@ public void process() { } catch (WorkloadException we) { terminateWorkload(); return; - } catch (InterruptedException e) { - terminateWorkload(); - return; - } catch (Exception e) { + } catch (InterruptedException e) { + terminateWorkload(); + return; + } catch (Exception e) { LOGGER.error("unexpected exception", e); terminateWorkload(); return; @@ -163,13 +166,13 @@ private void processWorkload() throws InterruptedException { workloadContext.setStopDate(new Date()); workloadContext.setCurrentStage(null); workloadContext.mergeErrorStatistics(); - for (StageContext stageContext : workloadContext.getStageRegistry() - .getAllItems()) { - if (stageContext.getState().equals(StageState.FAILED)) { - workloadContext.setState(FAILED); - return; - } - } + for (StageContext stageContext : workloadContext.getStageRegistry() + .getAllItems()) { + if (stageContext.getState().equals(StageState.FAILED)) { + workloadContext.setState(FAILED); + return; + } + } workloadContext.setState(FINISHED); } @@ -198,29 +201,29 @@ private void runStage(StageContext stageContext) throws InterruptedException { workloadContext.setCurrentStage(stageContext); if (stageName.equals("delay") && closuredelay > 0) { - executeDelay(stageContext, closuredelay); - } else { - executeStage(stageContext); - - long elapsedTime = System.currentTimeMillis() - startStamp; - - LOGGER.info("END WORK: {}, Time elapsed: {}", stageName, millisToHMS(elapsedTime)); - LOGGER.info("============================================"); - if(closuredelay > 0) - executeDelay(stageContext, closuredelay); - } - LOGGER.info("successfully ran stage {}", id); - } - - private void executeDelay(StageContext stageContext, int closuredelay) - throws InterruptedException { - - LOGGER.info("sleeping for " + closuredelay + " seconds..."); - stageContext.setState(StageState.SLEEPING); - Thread.sleep(closuredelay * 1000); - LOGGER.info("sleep complete."); - stageContext.setState(StageState.COMPLETED); - } + executeDelay(stageContext, closuredelay); + } else { + executeStage(stageContext); + + long elapsedTime = System.currentTimeMillis() - startStamp; + + LOGGER.info("END WORK: {}, Time elapsed: {}", stageName, millisToHMS(elapsedTime)); + LOGGER.info("============================================"); + if(closuredelay > 0) + executeDelay(stageContext, closuredelay); + } + LOGGER.info("successfully ran stage {}", id); + } + + private void executeDelay(StageContext stageContext, int closuredelay) + throws InterruptedException { + + LOGGER.info("sleeping for " + closuredelay + " seconds..."); + stageContext.setState(StageState.SLEEPING); + Thread.sleep(closuredelay * 1000); + LOGGER.info("sleep complete."); + stageContext.setState(StageState.COMPLETED); + } private void executeStage(StageContext stageContext) { StageRunner runner = createStageRunner(stageContext); @@ -232,7 +235,7 @@ private void executeStage(StageContext stageContext) { try { executor.invokeAll(Arrays.asList(callables)); } catch (InterruptedException ie) { - executeTrigger(trigger, false, wsId); + executeTrigger(trigger, false, wsId); throw new CancelledException(); // workload cancelled } runner.dispose(); // early dispose runner @@ -268,13 +271,13 @@ private void terminateWorkload() { workloadContext.setState(TERMINATED); LOGGER.info("successfully terminated workload {}", id); } - + private void executeTrigger(String trigger, boolean isEnable, String wsId) { - if (trigger == null || trigger.isEmpty()) - return; - TriggerRunner runner = new TriggerRunner(workloadContext.getDriverRegistry()); - runner.runTrigger(isEnable, trigger, wsId); - } + if (trigger == null || trigger.isEmpty()) + return; + TriggerRunner runner = new TriggerRunner(workloadContext.getDriverRegistry()); + runner.runTrigger(isEnable, trigger, wsId); + } public void cancel() { String id = workloadContext.getId(); @@ -311,11 +314,11 @@ private void cancelWorkload() { if (Thread.interrupted()) LOGGER.warn("get cancelled when canceling workload {}", id); try { - if (!executor.awaitTermination(5, TimeUnit.SECONDS) - && !executor.awaitTermination(5, TimeUnit.SECONDS)) - executor.shutdownNow(); + if (!executor.awaitTermination(5, TimeUnit.SECONDS) + && !executor.awaitTermination(5, TimeUnit.SECONDS)) + executor.shutdownNow(); if (!awaitTermination(5) && !awaitTermination(10) && !awaitTermination(30)) - LOGGER.warn("get cancelled when canceling workload {}", id); + LOGGER.warn("get cancelled when canceling workload {}", id); } catch (InterruptedException e) { executor.shutdownNow(); Thread.currentThread().interrupt(); @@ -332,7 +335,7 @@ private void cancelWorkload() { workloadContext.setState(CANCELLED); LOGGER.info("successfully cancelled workload {}", id); } - + private boolean awaitTermination(int seconds) { try { if (!executor.isTerminated()) { diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Aborter.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Aborter.java index ac154f2d..5d7e1ac0 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Aborter.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Aborter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; @@ -27,9 +27,9 @@ /** * The class encapsulates how to handle abort request/response, internally, it * issues command to abort driver, and get log from driver. - * + * * @author ywang19, qzheng7 - * + * */ class Aborter extends AbstractCommandTasklet { @@ -63,7 +63,7 @@ private void executeAbort() { @Override protected void handleResponse(AbortResponse response) { - Report report = new Report(); + Report report = new Report(); for (Metrics metrics : response.getReport()) report.addMetrics(metrics); context.setReport(report); diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractCommandTasklet.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractCommandTasklet.java index 558447dc..91f74be2 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractCommandTasklet.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractCommandTasklet.java @@ -1,86 +1,86 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.controller.tasklet; - -import org.codehaus.jackson.map.*; -import org.codehaus.jackson.map.DeserializationConfig.Feature; - -import com.intel.cosbench.controller.model.TaskContext; -import com.intel.cosbench.protocol.Response; - -/** - * The base class encapsulates commands through HTTP request. - * - * @author ywang19, qzheng7 - */ -abstract class AbstractCommandTasklet extends - AbstractHttpTasklet { - - private Class clazz; - protected long timeDrift = 0; /* real time drift between controller and driver */ - private static int tolerableTimeDrift = 300; /* tolerable time drift between controller and driver */ - - protected abstract void handleResponse(T response); - - public AbstractCommandTasklet(TaskContext context, Class clazz) { - super(context); - this.clazz = clazz; - } - - protected void initObjectMapper() { - ObjectMapper mapper = new ObjectMapper(); - DeserializationConfig config = mapper.copyDeserializationConfig(); - config.disable(Feature.FAIL_ON_UNKNOWN_PROPERTIES); - mapper.setDeserializationConfig(config); - context.setMapper(mapper); - } - - protected void issueCommand(String command) { - int count = 3; - long timeStamp = System.currentTimeMillis(); - while (--count >= 0) { - issueCommand(command, String.valueOf(timeStamp)); - if (Math.abs(timeDrift) < tolerableTimeDrift) - break; - timeStamp = System.currentTimeMillis() + timeDrift / 2; - } - LOGGER.info("time drift between controller and driver-{} is {} mSec", - getDriver().getName(), timeDrift); - if (count < 0) - LOGGER.warn("time drift is still longer than tolerable time drift {} mSec after 3 times of synchronization", - tolerableTimeDrift); - } - - protected void issueCommand(String command, String content) { - T response = null; - String body = issueHttpRequest(command, content); - try { - response = context.getMapper().readValue(body, clazz); - } catch (Exception e) { - LOGGER.error("cannot parse response body", e); - throw new TaskletException(); // mark termination - } - if (!response.isSucc()) { - String msg = "driver report error: HTTP {} - {}"; - LOGGER.error(msg, response.getCode(), response.getError()); - throw new TaskletException(); // mark termination - } - handleResponse(response); // specific response handling - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.controller.tasklet; + +import org.codehaus.jackson.map.*; +import org.codehaus.jackson.map.DeserializationConfig.Feature; + +import com.intel.cosbench.controller.model.TaskContext; +import com.intel.cosbench.protocol.Response; + +/** + * The base class encapsulates commands through HTTP request. + * + * @author ywang19, qzheng7 + */ +abstract class AbstractCommandTasklet extends + AbstractHttpTasklet { + + private Class clazz; + protected long timeDrift = 0; /* real time drift between controller and driver */ + private static int tolerableTimeDrift = 300; /* tolerable time drift between controller and driver */ + + protected abstract void handleResponse(T response); + + public AbstractCommandTasklet(TaskContext context, Class clazz) { + super(context); + this.clazz = clazz; + } + + protected void initObjectMapper() { + ObjectMapper mapper = new ObjectMapper(); + DeserializationConfig config = mapper.copyDeserializationConfig(); + config.disable(Feature.FAIL_ON_UNKNOWN_PROPERTIES); + mapper.setDeserializationConfig(config); + context.setMapper(mapper); + } + + protected void issueCommand(String command) { + int count = 3; + long timeStamp = System.currentTimeMillis(); + while (--count >= 0) { + issueCommand(command, String.valueOf(timeStamp)); + if (Math.abs(timeDrift) < tolerableTimeDrift) + break; + timeStamp = System.currentTimeMillis() + timeDrift / 2; + } + LOGGER.info("time drift between controller and driver-{} is {} mSec", + getDriver().getName(), timeDrift); + if (count < 0) + LOGGER.warn("time drift is still longer than tolerable time drift {} mSec after 3 times of synchronization", + tolerableTimeDrift); + } + + protected void issueCommand(String command, String content) { + T response = null; + String body = issueHttpRequest(command, content); + try { + response = context.getMapper().readValue(body, clazz); + } catch (Exception e) { + LOGGER.error("cannot parse response body", e); + throw new TaskletException(); // mark termination + } + if (!response.isSucc()) { + String msg = "driver report error: HTTP {} - {}"; + LOGGER.error(msg, response.getCode(), response.getError()); + throw new TaskletException(); // mark termination + } + handleResponse(response); // specific response handling + } + +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractHttpTasklet.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractHttpTasklet.java index 9bd57d3b..ad62a3ca 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractHttpTasklet.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractHttpTasklet.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,12 +12,13 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; import java.io.*; +import java.net.SocketException; import java.net.SocketTimeoutException; import org.apache.commons.lang.StringUtils; @@ -56,20 +57,32 @@ protected String issueHttpRequest(String command, String content) { HttpClient client = context.getHttpClient(); HttpPost request = prepareRequest(content, url); String body = null; + HttpResponse response = null; try { - HttpResponse response = client.execute(request); - body = fetchResponseBody(response); + response = client.execute(request); + } catch (SocketException se) { + LOGGER.error("fail to POST driver with socket connection error, will give up the task" , se); + throw new CancelledException(); } catch (SocketTimeoutException ste) { - LOGGER.error("fail to POST driver", ste); + LOGGER.error("fail to POST driver with url=" + url , ste); throw new TaskletException(); // mark termination } catch (ConnectTimeoutException cte) { - LOGGER.error("fail to POST driver", cte); + LOGGER.error("fail to POST driver with url=" + url, cte); throw new TaskletException(); // mark termination } catch (InterruptedIOException ie) { throw new CancelledException(); // task cancelled } catch (Exception e) { - LOGGER.error("fail to POST driver", e); + LOGGER.error("fail to POST driver with url=" + url, e); throw new TaskletException(); // mark termination + } finally { + if(response != null) { +// LOGGER.info("try to fetch response body"); + try{ + body = fetchResponseBody(response); + }catch(IOException ioe) { + LOGGER.error("fail to fetch response body", ioe); + } + } } return body; // HTTP response body retrieved } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractTasklet.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractTasklet.java index 588d4bba..4591896f 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractTasklet.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/AbstractTasklet.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Authenticator.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Authenticator.java index 820fcce3..af5c1dc7 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Authenticator.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Authenticator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; @@ -24,9 +24,9 @@ /** * The class encapsulates how to handle authentication request/response. - * + * * @author ywang19, qzheng7 - * + * */ class Authenticator extends AbstractCommandTasklet { diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Bootor.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Bootor.java index cfdf7b4d..e16e221e 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Bootor.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Bootor.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; @@ -28,9 +28,9 @@ /** * The class encapsulates how to handle boot request/response, internally, it * issues command to ping destination. - * + * * @author ywang19, qzheng7 - * + * */ class Bootor extends AbstractCommandTasklet { @@ -48,18 +48,18 @@ public void execute() { @Override protected void handleResponse(PingResponse response) { - long driverTime = 0; - DriverInfo driver = getDriver(); - try { - driverTime = Long.parseLong(response.getTimeStamp()); - } catch (NumberFormatException e) { - LOGGER.debug("time stamp of driver {} can not be formated", driver.getName()); - } + long driverTime = 0; + DriverInfo driver = getDriver(); + try { + driverTime = Long.parseLong(response.getTimeStamp()); + } catch (NumberFormatException e) { + LOGGER.debug("time stamp of driver {} can not be formated", driver.getName()); + } timeDrift = System.currentTimeMillis() - driverTime; - if (!StringUtils.equals(response.getName(), driver.getName())){ - String msg = "expetect driver name {} dose not match the real name {}"; - LOGGER.debug(msg, driver.getName(), response.getName()); - } + if (!StringUtils.equals(response.getName(), driver.getName())){ + String msg = "expetect driver name {} dose not match the real name {}"; + LOGGER.debug(msg, driver.getName(), response.getName()); + } } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Closer.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Closer.java index 668c742a..15b6a86a 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Closer.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Closer.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,11 +12,13 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; +import java.util.List; + import com.intel.cosbench.bench.*; import com.intel.cosbench.controller.model.TaskContext; import com.intel.cosbench.protocol.CloseResponse; @@ -25,9 +27,9 @@ * The class encapsulates how to handle close request/response, internally, it * issues command to close the workload on driver, and get report and log from * driver. - * + * * @author ywang19, qzheng7 - * + * */ class Closer extends AbstractCommandTasklet { @@ -55,6 +57,9 @@ protected void handleResponse(CloseResponse response) { context.setLog(response.getDriverLog()); context.setState(response.getState()); context.setErrorStatistics(response.getErrorStatistics()); - } + for(Metrics metrics : response.getWrReport()){ + context.getWrReport().add(metrics); + } + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Launcher.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Launcher.java index 7a32c494..ab806032 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Launcher.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Launcher.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; @@ -25,9 +25,9 @@ /** * The class encapsulates how to handle launch request/response, internally, it * issues command to start workload execution on driver. - * + * * @author ywang19, qzheng7 - * + * */ class Launcher extends AbstractCommandTasklet { diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Querier.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Querier.java index 67d733f7..16311ff5 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Querier.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Querier.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; @@ -29,9 +29,9 @@ /** * The class encapsulates how to handle query request/response, internally, it * issues command to query driver to get performance snapshot. - * + * * @author ywang19, qzheng7 - * + * */ class Querier extends AbstractCommandTasklet { @@ -45,9 +45,9 @@ protected void execute() { do { sleep(); try{ - issueCommand("query", id); + issueCommand("query", id); }catch(Exception tle) { - LOGGER.warn("some unexpected exception occurs when ping drivers, but it's ignorable.", tle); + LOGGER.warn("some unexpected exception occurs when ping drivers, but it's ignorable.", tle); } } while (!context.getState().equals(FINISHED)); } @@ -63,11 +63,11 @@ private void sleep() { @Override protected void handleResponse(QueryResponse response) { - if (response == null) { - LOGGER.warn("no response gets from driver"); - return; - } - + if (response == null) { + LOGGER.warn("no response gets from driver"); + return; + } + if (!response.isRunning()) context.setState(FINISHED); // stop querying Date time = response.getTime(); diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Submitter.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Submitter.java index 25bbb68e..962c34dd 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Submitter.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Submitter.java @@ -1,78 +1,78 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.controller.tasklet; - -import static com.intel.cosbench.model.TaskState.SUBMITTED; - -import com.intel.cosbench.config.*; -import com.intel.cosbench.config.castor.CastorConfigTools; -import com.intel.cosbench.controller.model.*; -import com.intel.cosbench.protocol.SubmitResponse; - -/** - * The class encapsulates how to handle boot request/response, internally, it - * issues command to ping destination. - * - * @author ywang19, qzheng7 - * - */ -class Submitter extends AbstractCommandTasklet { - - public Submitter(TaskContext context) { - super(context, SubmitResponse.class); - } - - @Override - public void execute() { - MissionWriter writer = CastorConfigTools.getMissionWriter(); - Mission mission = createMission(); - issueCommand("submit", writer.toXmlString(mission)); - context.setState(SUBMITTED); - } - - private Mission createMission() { - Mission mission = new Mission(); - SchedulePlan plan = context.getSchedule(); - Work work = plan.getWork(); - mission.setName(work.getName()); - mission.setWorkers(plan.getWorkers()); - mission.setOffset(plan.getOffset()); - mission.setInterval(work.getInterval()); - mission.setDivision(work.getDivision()); - mission.setRuntime(work.getRuntime()); - mission.setRampup(work.getRampup()); - mission.setRampdown(work.getRampdown()); - mission.setTotalOps(work.getTotalOps()); - mission.setTotalBytes(work.getTotalBytes()); - mission.setTotalWorkers(work.getWorkers()); - mission.setConfig(work.getConfig()); - mission.setAuth(work.getAuth()); - mission.setStorage(work.getStorage()); - mission.setOperations(work.getOperations()); - LOGGER.debug("controller work config is:" +work.getConfig()); - LOGGER.debug("controller mission config is: "+ mission.getConfig()); - return mission; - } - - @Override - protected void handleResponse(SubmitResponse response) { - String id = response.getId(); - context.setMissionId(id); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.controller.tasklet; + +import static com.intel.cosbench.model.TaskState.SUBMITTED; + +import com.intel.cosbench.config.*; +import com.intel.cosbench.config.castor.CastorConfigTools; +import com.intel.cosbench.controller.model.*; +import com.intel.cosbench.protocol.SubmitResponse; + +/** + * The class encapsulates how to handle boot request/response, internally, it + * issues command to ping destination. + * + * @author ywang19, qzheng7 + * + */ +class Submitter extends AbstractCommandTasklet { + + public Submitter(TaskContext context) { + super(context, SubmitResponse.class); + } + + @Override + public void execute() { + MissionWriter writer = CastorConfigTools.getMissionWriter(); + Mission mission = createMission(); + issueCommand("submit", writer.toXmlString(mission)); + context.setState(SUBMITTED); + } + + private Mission createMission() { + Mission mission = new Mission(); + SchedulePlan plan = context.getSchedule(); + Work work = plan.getWork(); + mission.setName(work.getName()); + mission.setWorkers(plan.getWorkers()); + mission.setOffset(plan.getOffset()); + mission.setInterval(work.getInterval()); + mission.setDivision(work.getDivision()); + mission.setRuntime(work.getRuntime()); + mission.setRampup(work.getRampup()); + mission.setRampdown(work.getRampdown()); + mission.setTotalOps(work.getTotalOps()); + mission.setTotalBytes(work.getTotalBytes()); + mission.setTotalWorkers(work.getWorkers()); + mission.setConfig(work.getConfig()); + mission.setAuth(work.getAuth()); + mission.setStorage(work.getStorage()); + mission.setOperations(work.getOperations()); + LOGGER.debug("controller work config is:" +work.getConfig()); + LOGGER.debug("controller mission config is: "+ mission.getConfig()); + return mission; + } + + @Override + protected void handleResponse(SubmitResponse response) { + String id = response.getId(); + context.setMissionId(id); + } + +} diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Tasklet.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Tasklet.java index 0c234be0..a981e435 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Tasklet.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Tasklet.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/TaskletException.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/TaskletException.java index 683aa455..2169a99e 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/TaskletException.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/TaskletException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Tasklets.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Tasklets.java index df43c5f4..88bacda0 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Tasklets.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Tasklets.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.controller.tasklet; @@ -78,13 +78,13 @@ public static List newAborters(TaskRegistry tasks) { } return result; } - + public static List newTriggers(String trigger, DriverRegistry registry, boolean option, String wsId) { - List result = new ArrayList(); + List result = new ArrayList(); for (DriverContext driver : registry) { result.add(new Trigger(driver, trigger, option, wsId)); } return result; - } + } } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Trigger.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Trigger.java index d0b76e85..14e3dfc4 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Trigger.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/Trigger.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.tasklet; import org.apache.commons.lang.StringUtils; @@ -7,17 +25,17 @@ public class Trigger extends TriggerHttpTasklet { - public Trigger(DriverContext driver, String trigger, boolean option, String wsId) { - super(driver, trigger, option, wsId); - } - + public Trigger(DriverContext driver, String trigger, boolean option, String wsId) { + super(driver, trigger, option, wsId); + } + @Override public void execute() { initHttpClient(); initObjectMapper(); String content = getCmdLine(); if (content == null || content.isEmpty()) - return; + return; issueCommand("trigger", content); try { closeHttpClient(); @@ -25,34 +43,34 @@ public void execute() { LOGGER.error("unexpected exception", e); } } - + private String getCmdLine() { - trigger.replace(" ", ""); - int idxLeft = StringUtils.indexOf(trigger, '('); - int idxRight = StringUtils.indexOf(trigger, ')'); - if (idxLeft < 3 || ( idxRight != trigger.length()-1) - || !StringUtils.substring(trigger, idxLeft-3, idxLeft).equals(".sh")){ - LOGGER.error("trigger format is illegal, it should be like trigger=\"*.sh(arg1, arg2,...)\""); - return null; - } - scriptName = StringUtils.left(trigger, idxLeft); - String argStr = StringUtils.substring(trigger, idxLeft+1, idxRight); - return isEnable ? ("enableTrigger," + scriptName + "," + argStr) - : ("killTrigger," + driver.getPidMapValue(scriptName) + "," + scriptName); - } - + trigger.replace(" ", ""); + int idxLeft = StringUtils.indexOf(trigger, '('); + int idxRight = StringUtils.indexOf(trigger, ')'); + if (idxLeft < 3 || ( idxRight != trigger.length()-1) + || !StringUtils.substring(trigger, idxLeft-3, idxLeft).equals(".sh")){ + LOGGER.error("trigger format is illegal, it should be like trigger=\"*.sh(arg1, arg2,...)\""); + return null; + } + scriptName = StringUtils.left(trigger, idxLeft); + String argStr = StringUtils.substring(trigger, idxLeft+1, idxRight); + return isEnable ? ("enableTrigger," + scriptName + "," + argStr) + : ("killTrigger," + driver.getPidMapValue(scriptName) + "," + scriptName); + } + @Override protected void handleResponse(TriggerResponse response) { driver.putPidMap(scriptName, response.getPID()); - String log = response.getScriptLog(); - if (log == null || log.isEmpty()) - LOGGER.warn("no log for {} on {}", (isEnable ? "enable ":"kill ") + scriptName, driver.getName()); - if (!isEnable) { - String enableLog = driver.getLogMapValue(wsId); - driver.putLogMap(wsId, enableLog + log); - return; - } - driver.putLogMap(wsId, scriptName + ";" + log); + String log = response.getScriptLog(); + if (log == null || log.isEmpty()) + LOGGER.warn("no log for {} on {}", (isEnable ? "enable ":"kill ") + scriptName, driver.getName()); + if (!isEnable) { + String enableLog = driver.getLogMapValue(wsId); + driver.putLogMap(wsId, enableLog + log); + return; + } + driver.putLogMap(wsId, scriptName + ";" + log); } - + } diff --git a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/TriggerHttpTasklet.java b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/TriggerHttpTasklet.java index c93cad6e..67b75f16 100644 --- a/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/TriggerHttpTasklet.java +++ b/dev/cosbench-controller/src/com/intel/cosbench/controller/tasklet/TriggerHttpTasklet.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.controller.tasklet; import java.io.IOException; @@ -24,7 +42,7 @@ import com.intel.cosbench.service.UnexpectedException; abstract class TriggerHttpTasklet implements Tasklet{ - private transient HttpClient httpClient; + private transient HttpClient httpClient; private transient ObjectMapper mapper; protected DriverContext driver; protected String trigger = null; @@ -32,21 +50,21 @@ abstract class TriggerHttpTasklet implements Tasklet{ protected String scriptName = null; protected String wsId = null; - Class clazz = TriggerResponse.class; - + Class clazz = TriggerResponse.class; + private static final int TIMEOUT = 300 * 1000; protected static final Logger LOGGER = LogFactory.getSystemLogger(); - + protected abstract void execute(); protected abstract void handleResponse(TriggerResponse response); - + public TriggerHttpTasklet(DriverContext driver, String trigger, boolean option, String wsId) { - this.driver = driver; - this.trigger = trigger; - this.isEnable = option; - this.wsId = wsId; - } - + this.driver = driver; + this.trigger = trigger; + this.isEnable = option; + this.wsId = wsId; + } + protected void initObjectMapper() { ObjectMapper mapper = new ObjectMapper(); DeserializationConfig config = mapper.copyDeserializationConfig(); @@ -117,13 +135,13 @@ private static String fetchResponseBody(HttpResponse response) LOGGER.debug("[ << ] - {} [body-omitted]", status); return body; // the response body } - + protected void issueCommand(String command, String content) { TriggerResponse response = null; String body = issueHttpRequest(command, content); if (body == null) { - LOGGER.error("TriggerResponse body is null"); - return; + LOGGER.error("TriggerResponse body is null"); + return; } try { response = this.mapper.readValue(body, clazz); @@ -138,11 +156,11 @@ protected void issueCommand(String command, String content) { } handleResponse(response); } - + @Override public Tasklet call() { try { - execute(); + execute(); } catch (Exception e) { LOGGER.error("unexpected exception of trigger", e); } diff --git a/dev/cosbench-core-web/META-INF/MANIFEST.MF b/dev/cosbench-core-web/META-INF/MANIFEST.MF index f4407b82..58fb986b 100644 --- a/dev/cosbench-core-web/META-INF/MANIFEST.MF +++ b/dev/cosbench-core-web/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Web Core Bundle Bundle-SymbolicName: cosbench-core-web -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Export-Package: com.intel.cosbench.web diff --git a/dev/cosbench-core-web/src/com/intel/cosbench/web/AbstractController.java b/dev/cosbench-core-web/src/com/intel/cosbench/web/AbstractController.java index 6e287f84..10372ee1 100644 --- a/dev/cosbench-core-web/src/com/intel/cosbench/web/AbstractController.java +++ b/dev/cosbench-core-web/src/com/intel/cosbench/web/AbstractController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.web; @@ -27,7 +27,7 @@ /** * The base class encapsulates spring mvc controller. - * + * * @author ywang19, qzheng7 * */ @@ -47,7 +47,7 @@ public ModelAndView handleRequest(HttpServletRequest req, } catch (NotFoundException nfe) { return new ModelAndView("404"); } catch (FileNotFoundException fnfe) { - return new ModelAndView("404", "resource", fnfe.getMessage()); + return new ModelAndView("404", "resource", fnfe.getMessage()); } catch (Exception e) { StringWriter writer = new StringWriter(); e.printStackTrace(new PrintWriter(writer)); diff --git a/dev/cosbench-core-web/src/com/intel/cosbench/web/BadRequestException.java b/dev/cosbench-core-web/src/com/intel/cosbench/web/BadRequestException.java index 6cff943f..d132371b 100644 --- a/dev/cosbench-core-web/src/com/intel/cosbench/web/BadRequestException.java +++ b/dev/cosbench-core-web/src/com/intel/cosbench/web/BadRequestException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.web; diff --git a/dev/cosbench-core-web/src/com/intel/cosbench/web/NotFoundException.java b/dev/cosbench-core-web/src/com/intel/cosbench/web/NotFoundException.java index c69eb45d..6830da1e 100644 --- a/dev/cosbench-core-web/src/com/intel/cosbench/web/NotFoundException.java +++ b/dev/cosbench-core-web/src/com/intel/cosbench/web/NotFoundException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.web; diff --git a/dev/cosbench-core/META-INF/MANIFEST.MF b/dev/cosbench-core/META-INF/MANIFEST.MF index 227c2a14..d662447c 100644 --- a/dev/cosbench-core/META-INF/MANIFEST.MF +++ b/dev/cosbench-core/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Core Bundle Bundle-SymbolicName: cosbench-core -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Export-Package: com.intel.cosbench.bench, diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Aggregator.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Aggregator.java index a5dc62a4..66407fd3 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Aggregator.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Aggregator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; @@ -23,9 +23,9 @@ import java.util.*; /** - * This class encapsulates operations to aggregate data from all drivers, + * This class encapsulates operations to aggregate data from all drivers, * also includes operations to calculate response time distribution. - * + * * @author ywang19, qzheng7 * */ @@ -67,8 +67,8 @@ public Metrics aggregate() { metrics.setAvgResTime(getAvgResTime()); metrics.setAvgXferTime(getAvgXferTime()); metrics.setLatency(getLatency()); - metrics.setRatio(metrics.getTotalSampleCount() > 0 ? (double) metrics - .getSampleCount() / metrics.getTotalSampleCount() : 0D); + metrics.setRatio(metrics.getTotalSampleCount() > 0 ? (double) metrics + .getSampleCount() / metrics.getTotalSampleCount() : 0D); return metrics; } @@ -96,13 +96,13 @@ private double getAvgResTime() { } private double getAvgXferTime() { - if (sampleCount == 0) - return 0D; - double sum = 0D; - for (Metrics metrics : children) - sum += metrics.getAvgXferTime() * metrics.getSampleCount(); - return sum / sampleCount; - } + if (sampleCount == 0) + return 0D; + double sum = 0D; + for (Metrics metrics : children) + sum += metrics.getAvgXferTime() * metrics.getSampleCount(); + return sum / sampleCount; + } private Histogram getLatency() { if (!containsLatency) diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Benchmark.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Benchmark.java index b0dd3044..91978714 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Benchmark.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Benchmark.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Counter.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Counter.java index d6a79bbc..7d3c8c40 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Counter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Counter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,17 +12,17 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; import java.util.concurrent.atomic.AtomicInteger; /** - * The class encapsulates the sample counting of response time histogram, the buckets are selected by using + * The class encapsulates the sample counting of response time histogram, the buckets are selected by using * 10 milliseconds as interval and 500 seconds as upper limit. - * + * * @author ywang19, qzheng7 * */ @@ -33,7 +33,7 @@ public class Counter { /* limit for the response time histogram */ public final static long RES_MAX = 500 * 1000; // 500 seconds - + /* the index of upper limit */ public final static int UL = (int) (RES_MAX/RES_INT); @@ -56,8 +56,8 @@ public int get(int index) { /** * The method counts the time to corresponding bucket. - * - * @param time the response time to be counted + * + * @param time the response time to be counted * @return */ public void doAdd(long time) { diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/ErrorStatistics.java b/dev/cosbench-core/src/com/intel/cosbench/bench/ErrorStatistics.java index 7ab2efd5..ca1fcb98 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/ErrorStatistics.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/ErrorStatistics.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; import java.util.HashMap; @@ -25,104 +25,107 @@ import com.intel.cosbench.log.Logger; public class ErrorStatistics { - private HashMap stackTraceAndException; - private HashMap stackTraceAndTargets; - - //summary the result - private HashMap stackTraceAndMessage; - private HashMap stackTraceAndErrorCode; - private HashMap stackTraceAndNum; - private HashMap stackTraceAndEntireTrace; - private HashMap errorCodeAndNum; - - - public ErrorStatistics(){ - stackTraceAndException = new HashMap(); - stackTraceAndTargets = new HashMap(); - stackTraceAndMessage = new HashMap(); - stackTraceAndErrorCode = new HashMap(); - stackTraceAndEntireTrace = new HashMap(); - stackTraceAndNum = new HashMap(); - errorCodeAndNum = new HashMap(); - } - - - public HashMap getStackTraceAndException() { - return stackTraceAndException; - } - - public HashMap getStackTraceAndTargets() { - return stackTraceAndTargets; - } - - - public HashMap getStackTraceAndMessage() { - return stackTraceAndMessage; - } - - - public HashMap getStackTraceAndErrorCode() { - return stackTraceAndErrorCode; - } - - - public HashMap getStackTraceAndEntireTrace() { - return stackTraceAndEntireTrace; - } - - - - public HashMap getStackTraceAndNum() { - return stackTraceAndNum; - } - - public HashMap getErrorCodeAndNum() { - return errorCodeAndNum; - } - - - public void summaryToMission(Logger logger){ - Exception e = null; - String message = null; - String code = null; - Integer codeNumber; - String trace = ""; - for(Map.Entry entry : stackTraceAndTargets.entrySet()){ - e = stackTraceAndException.get(entry.getKey()); - if (e != null) - message = e.getMessage(); - stackTraceAndMessage.put(entry.getKey(), message); - if (message != null) - code = message.substring(9, 12); - if(!isInteger(code)) - code = "N/A"; - stackTraceAndErrorCode.put(entry.getKey(), code); - codeNumber = getCodeNumber(entry.getValue()); - stackTraceAndNum.put(entry.getKey(), String.valueOf(codeNumber)); - if(errorCodeAndNum.containsKey(code)){ - Integer num = errorCodeAndNum.get(code); - errorCodeAndNum.put(code, num + codeNumber); - } - else{ - errorCodeAndNum.put(code, codeNumber); - } - for(StackTraceElement ele: stackTraceAndException.get(entry.getKey()).getStackTrace()) - trace += ele.toString()+"\n"; - stackTraceAndEntireTrace.put(entry.getKey(), trace); - logger.error("error code: " + code + " occurred " +codeNumber + " times, fail to operate: " + entry.getValue(), stackTraceAndException.get(entry.getKey())); - } - } - - public Integer getCodeNumber(String targets){ - if (targets == null) - return null; - int codeNumber = targets.split(",").length; - return codeNumber; - } - - public static boolean isInteger(String str) { - Pattern pattern = Pattern.compile("^[-\\+]?[\\d]*$"); - return pattern.matcher(str).matches(); - } + private HashMap stackTraceAndException; + private HashMap stackTraceAndTargets; + + //summary the result + private HashMap stackTraceAndMessage; + private HashMap stackTraceAndErrorCode; + private HashMap stackTraceAndNum; + private HashMap stackTraceAndEntireTrace; + private HashMap errorCodeAndNum; + + + public ErrorStatistics(){ + stackTraceAndException = new HashMap(); + stackTraceAndTargets = new HashMap(); + stackTraceAndMessage = new HashMap(); + stackTraceAndErrorCode = new HashMap(); + stackTraceAndEntireTrace = new HashMap(); + stackTraceAndNum = new HashMap(); + errorCodeAndNum = new HashMap(); + } + + + public HashMap getStackTraceAndException() { + return stackTraceAndException; + } + + public HashMap getStackTraceAndTargets() { + return stackTraceAndTargets; + } + + + public HashMap getStackTraceAndMessage() { + return stackTraceAndMessage; + } + + + public HashMap getStackTraceAndErrorCode() { + return stackTraceAndErrorCode; + } + + + public HashMap getStackTraceAndEntireTrace() { + return stackTraceAndEntireTrace; + } + + + + public HashMap getStackTraceAndNum() { + return stackTraceAndNum; + } + + public HashMap getErrorCodeAndNum() { + return errorCodeAndNum; + } + + + public void summaryToMission(Logger logger){ + Exception e = null; + String message = null; + String code = null; + Integer codeNumber; + String trace = ""; + for(Map.Entry entry : stackTraceAndTargets.entrySet()){ + e = stackTraceAndException.get(entry.getKey()); + if (e != null) + message = e.getMessage(); + stackTraceAndMessage.put(entry.getKey(), message); + if (message != null) + code = message.substring(9, 12); + if(!isInteger(code)) + code = "N/A"; + stackTraceAndErrorCode.put(entry.getKey(), code); + codeNumber = getCodeNumber(entry.getValue()); + stackTraceAndNum.put(entry.getKey(), String.valueOf(codeNumber)); + if(errorCodeAndNum.containsKey(code)){ + Integer num = errorCodeAndNum.get(code); + errorCodeAndNum.put(code, num + codeNumber); + } + else{ + errorCodeAndNum.put(code, codeNumber); + } + for(StackTraceElement ele: stackTraceAndException.get(entry.getKey()).getStackTrace()) + trace += ele.toString()+"\n"; + stackTraceAndEntireTrace.put(entry.getKey(), trace); + logger.error("error code: " + code + " occurred " +codeNumber + " times, fail to operate: " + entry.getValue(), stackTraceAndException.get(entry.getKey())); + } + } + + public Integer getCodeNumber(String targets){ + if (targets == null) + return null; + int codeNumber = targets.split(",").length; + return codeNumber; + } + + public static boolean isInteger(String str) { + Pattern pattern = Pattern.compile("^[-\\+]?[\\d]*$"); + if(str !=null && pattern.matcher(str) != null) + return pattern.matcher(str).matches(); + else + return false; + } } diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Histogram.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Histogram.java index af1e44df..272f6ffc 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Histogram.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Histogram.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,15 +12,15 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; /** - * This class encapsulates calculation of different percentiles, + * This class encapsulates calculation of different percentiles, * so far it supports 60/80/90/95/99/100 percentiles. - * + * * @author ywang19, qzheng7 * */ @@ -101,9 +101,9 @@ public void setHistoData(int[] histoData) { /** * The method calculates 60/80/90/95/99 percentiles. - * - * @param - * @return + * + * @param + * @return */ public void recalcPercentiles() { int curr = 0; diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Mark.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Mark.java index 760523e4..5de2d202 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Mark.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Mark.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; @@ -22,7 +22,7 @@ /** * The class aggregates performance data in one sample interval. - * + * * @author ywang19, qzheng7 * */ @@ -64,21 +64,21 @@ public String getOpType() { public void setOpType(String opType) { this.opType = opType; } - + public void setOpName(String opName){ - this.opName = opName; + this.opName = opName; } - + public String getOpName(){ - return opName; + return opName; } - + public void setOpId(String opId) { - this.opId = opId; + this.opId = opId; } - + public String getOpId() { - return opId; + return opId; } public String getSampleType() { @@ -130,12 +130,12 @@ public void setRtSum(long rtSum) { } public long getXtSum() { - return xtSum; - } + return xtSum; + } public void setXtSum(long xtSum) { - this.xtSum = xtSum; - } + this.xtSum = xtSum; + } public long getByteCount() { return byteCount; @@ -163,7 +163,7 @@ public void addSample(Sample sample) { xtSum += sample.getXferTime(); byteCount += sample.getBytes(); } - + totalSampleCount += 1; } @@ -173,10 +173,10 @@ public void addOperation(Result result) { totalOpCount += 1; } - public static String getMarkType(String opId, String opType, - String sampleType, String opName) { - return opId + "." + opType + "." + sampleType + "." + opName; - } + public static String getMarkType(String opId, String opType, + String sampleType, String opName) { + return opId + "." + opType + "." + sampleType + "." + opName; + } public static Mark newMark(String type) { String[] types = type.split("\\."); diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Metrics.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Metrics.java index b1fa5c96..08e99b40 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Metrics.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Metrics.java @@ -1,232 +1,231 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.bench; - -import com.intel.cosbench.utils.MapRegistry.Item; - - -/** - * The class represents the overall performance metrics per each type. - * - * @author ywang19, qzheng7 - * - */ -public class Metrics implements Item, Cloneable { - - private String name; /* metrics id */ - - /* Type */ - - private String opType; /* operation type */ - private String sampleType; /* sample type */ - private String opName; /* operation name*/ - private String opId; /* operation id */ - - /* Status */ - - private int sampleCount; /* number of successful samples */ - private int totalSampleCount; /* total operations issued */ - private long byteCount; /* total bytes transferred */ - private int workerCount; /* total workers involved */ - - /* Metrics */ - - private double avgResTime; /* average response time */ - private double avgXferTime; /* average transfer time */ - private double throughput; /* operation throughput */ - private double bandwidth; /* network bandwidth */ - - /* Latency Details */ - private Histogram latency; /* detailed latency metrics */ - - /* success ratio */ - private double ratio; - - public Metrics() { - /* empty */ - } - - @Override - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public String getOpType() { - return opType; - } - - public void setOpType(String opType) { - this.opType = opType; - } - - public String getSampleType() { - return sampleType; - } - - public void setSampleType(String sampleType) { - this.sampleType = sampleType; - } - - public String getOpName(){ - return opName; - } - - public void setOpName(String opName){ - this.opName = opName; - } - - public String getOpId() { - return opId; - } - - public void setOpId(String opId) { - this.opId = opId; - } - - public int getSampleCount() { - return sampleCount; - } - - public void setSampleCount(int sampleCount) { - this.sampleCount = sampleCount; - } - - public int getTotalSampleCount() { - return totalSampleCount; - } - - public void setTotalSampleCount(int totalSampleCount) { - this.totalSampleCount = totalSampleCount; - } - - public long getByteCount() { - return byteCount; - } - - public void setByteCount(long byteCount) { - this.byteCount = byteCount; - } - - public int getWorkerCount() { - return workerCount; - } - - public void setWorkerCount(int workerCount) { - this.workerCount = workerCount; - } - - public double getAvgResTime() { - return avgResTime; - } - - public void setAvgResTime(double avgResTime) { - this.avgResTime = avgResTime; - } - - public double getAvgXferTime() { - return avgXferTime; - } - - public void setAvgXferTime(double avgXferTime) { - this.avgXferTime = avgXferTime; - } - - public double getThroughput() { - return throughput; - } - - public void setThroughput(double throughput) { - this.throughput = throughput; - } - - public double getBandwidth() { - return bandwidth; - } - - public void setBandwidth(double bandwidth) { - this.bandwidth = bandwidth; - } - - public Histogram getLatency() { - return latency; - } - - public void setLatency(Histogram latency) { - this.latency = latency; - } - - public void setRatio(double ratio) { - this.ratio = ratio; - } - - public double getRatio() { - return ratio; - } - - @Override - public Metrics clone() { - try { - return (Metrics) super.clone(); - } catch (CloneNotSupportedException e) { - } - return this; - } - - public static String getMetricsType(String opId, String opType, - String sampleType, String opName) { - return opId + "." + opType + "." + sampleType + "." + opName; - } - - public static Metrics newMetrics(String type) { - String[] types = type.split("\\."); - Metrics metrics = new Metrics(); - metrics.setName(type); - metrics.setOpId(types.length > 0? types[0] : "na"); - metrics.setOpType(types.length > 1? types[1] : "na"); - metrics.setSampleType(types.length > 2? types[2] : "na"); - metrics.setOpName(types.length > 3? types[3] : "na"); - return metrics; - } - - public static Metrics convert(Mark mark, long window) { - int sps = mark.getSampleCount(); - int tsps = mark.getTotalSampleCount(); - long rtSum = mark.getRtSum(); - long xtSum = mark.getXtSum(); - long bytes = mark.getByteCount(); - String type = getMetricsType(mark.getOpId(), mark.getOpType(), - mark.getSampleType(), mark.getOpName()); - Metrics metrics = newMetrics(type); - metrics.setSampleCount(sps); - metrics.setTotalSampleCount(tsps); - metrics.setRatio(metrics.getTotalSampleCount() > 0 ? (double) metrics - .getSampleCount() / metrics.getTotalSampleCount() : 0D); - metrics.setByteCount(bytes); - metrics.setWorkerCount(1); - metrics.setAvgResTime(rtSum > 0 ? ((double) rtSum) / sps : 0); - metrics.setAvgXferTime(xtSum > 0 ? ((double) xtSum) / sps : 0); - metrics.setThroughput(sps > 0 ? ((double) sps) / window * 1000 : 0); - metrics.setBandwidth(bytes > 0 ? ((double) bytes) / window * 1000 : 0); - return metrics; - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.bench; + +import com.intel.cosbench.utils.MapRegistry.Item; + + +/** + * The class represents the overall performance metrics per each type. + * + * @author ywang19, qzheng7 + * + */ +public class Metrics implements Item, Cloneable { + + private String name; /* metrics id */ + + /* Type */ + + private String opType; /* operation type */ + private String sampleType; /* sample type */ + private String opName; /* operation name*/ + private String opId; /* operation id */ + + /* Status */ + + private int sampleCount; /* number of successful samples */ + private int totalSampleCount; /* total operations issued */ + private long byteCount; /* total bytes transferred */ + private int workerCount; /* total workers involved */ + + /* Metrics */ + + private double avgResTime; /* average response time */ + private double avgXferTime; /* average transfer time */ + private double throughput; /* operation throughput */ + private double bandwidth; /* network bandwidth */ + + /* Latency Details */ + private Histogram latency; /* detailed latency metrics */ + + /* success ratio */ + private double ratio; + + public Metrics() { + /* empty */ + } + + @Override + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public String getOpType() { + return opType; + } + + public void setOpType(String opType) { + this.opType = opType; + } + + public String getSampleType() { + return sampleType; + } + + public void setSampleType(String sampleType) { + this.sampleType = sampleType; + } + + public String getOpName(){ + return opName; + } + + public void setOpName(String opName){ + this.opName = opName; + } + + public String getOpId() { + return opId; + } + + public void setOpId(String opId) { + this.opId = opId; + } + + public int getSampleCount() { + return sampleCount; + } + + public void setSampleCount(int sampleCount) { + this.sampleCount = sampleCount; + } + + public int getTotalSampleCount() { + return totalSampleCount; + } + + public void setTotalSampleCount(int totalSampleCount) { + this.totalSampleCount = totalSampleCount; + } + + public long getByteCount() { + return byteCount; + } + + public void setByteCount(long byteCount) { + this.byteCount = byteCount; + } + + public int getWorkerCount() { + return workerCount; + } + + public void setWorkerCount(int workerCount) { + this.workerCount = workerCount; + } + + public double getAvgResTime() { + return avgResTime; + } + + public void setAvgResTime(double avgResTime) { + this.avgResTime = avgResTime; + } + + public double getAvgXferTime() { + return avgXferTime; + } + + public void setAvgXferTime(double avgXferTime) { + this.avgXferTime = avgXferTime; + } + + public double getThroughput() { + return throughput; + } + + public void setThroughput(double throughput) { + this.throughput = throughput; + } + + public double getBandwidth() { + return bandwidth; + } + + public void setBandwidth(double bandwidth) { + this.bandwidth = bandwidth; + } + + public Histogram getLatency() { + return latency; + } + + public void setLatency(Histogram latency) { + this.latency = latency; + } + + public void setRatio(double ratio) { + this.ratio = ratio; + } + + public double getRatio() { + return ratio; + } + + @Override + public Metrics clone() { + try { + return (Metrics) super.clone(); + } catch (CloneNotSupportedException e) { + } + return this; + } + + public static String getMetricsType(String opId, String opType, + String sampleType, String opName) { + return opId + "." + opType + "." + sampleType + "." + opName; + } + + public static Metrics newMetrics(String type) { + String[] types = type.split("\\."); + Metrics metrics = new Metrics(); + metrics.setName(type); + metrics.setOpId(types.length > 0? types[0] : "na"); + metrics.setOpType(types.length > 1? types[1] : "na"); + metrics.setSampleType(types.length > 2? types[2] : "na"); + metrics.setOpName(types.length > 3? types[3] : "na"); + return metrics; + } + + public static Metrics convert(Mark mark, long window) { + int sps = mark.getSampleCount(); + int tsps = mark.getTotalSampleCount(); + long rtSum = mark.getRtSum(); + long xtSum = mark.getXtSum(); + long bytes = mark.getByteCount(); + String type = getMetricsType(mark.getOpId(), mark.getOpType(), + mark.getSampleType(), mark.getOpName()); + Metrics metrics = newMetrics(type); + metrics.setSampleCount(sps); + metrics.setTotalSampleCount(tsps); + metrics.setRatio(metrics.getTotalSampleCount() > 0 ? (double) metrics + .getSampleCount() / metrics.getTotalSampleCount() : 0D); + metrics.setByteCount(bytes); + metrics.setWorkerCount(1); + metrics.setAvgResTime(rtSum > 0 ? ((double) rtSum) / sps : 0); + metrics.setAvgXferTime(xtSum > 0 ? ((double) xtSum) / sps : 0); + metrics.setThroughput(sps > 0 ? ((double) sps) / window * 1000 : 0); + metrics.setBandwidth(bytes > 0 ? ((double) bytes) / window * 1000 : 0); + return metrics; + } +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Report.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Report.java index 43b3196d..cf0b4603 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Report.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Report.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; @@ -22,7 +22,7 @@ /** * This class is a list of metrics, it represents all metrics needed for one report. - * + * * @author ywang19, qzheng7 * @see com.intel.cosbench.bench.Metrics * @@ -41,4 +41,5 @@ public Metrics[] getAllMetrics() { return getAllItems().toArray(new Metrics[getSize()]); } + } diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/ReportMerger.java b/dev/cosbench-core/src/com/intel/cosbench/bench/ReportMerger.java index 93dc2556..0e7e089f 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/ReportMerger.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/ReportMerger.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; @@ -22,7 +22,7 @@ /** * This class encapsulates operations needed to merge metrics into report. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Result.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Result.java index e60de325..68c46612 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Result.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Result.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; @@ -21,7 +21,7 @@ /** * This class encapsulates the returned result for one operation. - * + * * @author ywang19, qzheng7 * */ @@ -35,15 +35,15 @@ public class Result { private String opName; private String opId; - public Result(Date timestamp, String opId, String opType, - String sampleType, String opName, boolean succ) { - this.timestamp = timestamp; - this.succ = succ; - this.opType = opType; - this.sampleType = sampleType; - this.opName = opName; - this.opId = opId; - } + public Result(Date timestamp, String opId, String opType, + String sampleType, String opName, boolean succ) { + this.timestamp = timestamp; + this.succ = succ; + this.opType = opType; + this.sampleType = sampleType; + this.opName = opName; + this.opId = opId; + } public Date getTimestamp() { return timestamp; @@ -76,13 +76,13 @@ public String getSampleType() { public void setSampleType(String sampleType) { this.sampleType = sampleType; } - + public String getOpName(){ - return opName; + return opName; } - + public String getOpId() { - return opId; + return opId; } } diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Sample.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Sample.java index 63286218..bacb39a7 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Sample.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Sample.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; @@ -21,7 +21,7 @@ /** * This class encapsulates the performance sample of one operation. - * + * * @author ywang19, qzheng7 * */ @@ -39,36 +39,36 @@ public class Sample { private long xferTime; /* transfer time */ private long bytes; /* bytes transferred */ - public Sample(Date timestamp, String opId, String opType, - String sampleType, String opName, boolean succ) { - this(timestamp, opId, opType, sampleType, opName, succ, 0L, 0L, 0L); - } - - public Sample(Date timestamp, String opId, String opType, - String sampleType, String opName, boolean succ, long time, long xferTime, - long bytes) { - this.timestamp = timestamp; - this.succ = succ; - this.time = time; - this.xferTime = xferTime; - this.bytes = bytes; - this.opType = opType; - this.sampleType = sampleType; - this.opName = opName; - this.opId = opId; - } + public Sample(Date timestamp, String opId, String opType, + String sampleType, String opName, boolean succ) { + this(timestamp, opId, opType, sampleType, opName, succ, 0L, 0L, 0L); + } + + public Sample(Date timestamp, String opId, String opType, + String sampleType, String opName, boolean succ, long time, long xferTime, + long bytes) { + this.timestamp = timestamp; + this.succ = succ; + this.time = time; + this.xferTime = xferTime; + this.bytes = bytes; + this.opType = opType; + this.sampleType = sampleType; + this.opName = opName; + this.opId = opId; + } public Date getTimestamp() { return timestamp; } - - public String getOpName() { - return opName; - } - public String getOpId() { - return opId; - } + public String getOpName() { + return opName; + } + + public String getOpId() { + return opId; + } public void setTimestamp(Date timestamp) { this.timestamp = timestamp; @@ -107,12 +107,12 @@ public void setTime(long time) { } public long getXferTime() { - return xferTime; - } + return xferTime; + } public void setXferTime(long xferTime) { - this.xferTime = xferTime; - } + this.xferTime = xferTime; + } public long getBytes() { return bytes; diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Snapshot.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Snapshot.java index 15aa7b87..b4e172be 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Snapshot.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Snapshot.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; @@ -22,7 +22,7 @@ /** * This class represents the aggregated (from different drivers) performance data at one sample interval. - * + * * @author ywang19, qzheng7 * */ @@ -51,7 +51,7 @@ public Snapshot(Report report, Date timestamp) { public Report getReport() { return report; } - + public Date getTimestamp() { return timestamp; } diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/SnapshotMerger.java b/dev/cosbench-core/src/com/intel/cosbench/bench/SnapshotMerger.java index ec1df3a1..54675f29 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/SnapshotMerger.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/SnapshotMerger.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; /** * This class encapsulates operations needed to merge report to snapshot. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/Status.java b/dev/cosbench-core/src/com/intel/cosbench/bench/Status.java index edaa56f1..7f5b599f 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/bench/Status.java +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/Status.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.bench; @@ -21,7 +21,7 @@ /** * This class is a list of mark, it represents all marks at different time point. - * + * * @author ywang19, qzheng7 * @see com.intel.cosbench.bench.Metrics * diff --git a/dev/cosbench-core/src/com/intel/cosbench/bench/TaskReport.java b/dev/cosbench-core/src/com/intel/cosbench/bench/TaskReport.java new file mode 100644 index 00000000..2cce00c1 --- /dev/null +++ b/dev/cosbench-core/src/com/intel/cosbench/bench/TaskReport.java @@ -0,0 +1,49 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.bench; +/** + * The class is the data structure of task info + * + * @author liyuan + * + */ +public class TaskReport { + private String driverName; + private String driverUrl; + private Report report; + public String getDriverName() { + return driverName; + } + public void setDriverName(String driverName) { + this.driverName = driverName; + } + public String getDriverUrl() { + return driverUrl; + } + public void setDriverUrl(String driverUrl) { + this.driverUrl = driverUrl; + } + public Report getReport() { + return report; + } + public void setReport(Report report) { + this.report = report; + } + +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractLatencyExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractLatencyExporter.java index b7771a47..ce15eff7 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractLatencyExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractLatencyExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -24,7 +24,7 @@ /** * This class is the base class for exporting response time histogram. - * + * * @author ywang19, qzheng7 * */ @@ -56,7 +56,7 @@ public void export(Writer writer) throws IOException { for (Metrics metrics : workload.getReport()) { int sum = 0; if(metrics.getLatency() == null) - continue; + continue; int[] data = metrics.getLatency().getHistoData(); for (int idx = 0; idx < size; idx++) sum += data[idx]; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractMatrixExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractMatrixExporter.java index d3df2f4e..892d2b26 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractMatrixExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractMatrixExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -24,7 +24,7 @@ /** * This class is the base class for exporting performance matrix. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractRunExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractRunExporter.java index 37880480..f87e85ea 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractRunExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractRunExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -23,7 +23,7 @@ /** * This class is the base class for exporting run information. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractStageExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractStageExporter.java index 9442175d..a5a57ed9 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractStageExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractStageExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -24,7 +24,7 @@ /** * This class is the base class for exporting stage information. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractTaskExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractTaskExporter.java new file mode 100644 index 00000000..5e67e5b3 --- /dev/null +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractTaskExporter.java @@ -0,0 +1,80 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.exporter; + +import java.io.IOException; +import java.io.Writer; +import java.util.ArrayList; +import java.util.List; + +import com.intel.cosbench.bench.Snapshot; +import com.intel.cosbench.bench.TaskReport; +import com.intel.cosbench.model.DriverInfo; +import com.intel.cosbench.model.StageInfo; +import com.intel.cosbench.model.WorkloadInfo; + +public abstract class AbstractTaskExporter implements TaskExporter { + + protected WorkloadInfo workloadInfo; + protected DriverInfo driverInfo; + protected List driverTasks = new ArrayList(); + + public WorkloadInfo getWorkloadInfo() { + return workloadInfo; + } + + public void setWorkloadInfo(WorkloadInfo workloadInfo) { + this.workloadInfo = workloadInfo; + } + + public DriverInfo getDriverInfo() { + return driverInfo; + } + + public void setDriverInfo(DriverInfo driverInfo) { + this.driverInfo = driverInfo; + } + + + @Override + public void export(Writer writer) throws IOException { + //check every stage every task + for(StageInfo sInfo:workloadInfo.getStageInfos()){ + for(TaskReport tReport:sInfo.getTaskReports()){ + if(tReport.getDriverUrl().equals(driverInfo.getUrl())){ + driverTasks.add(tReport); + } + } + } + writeReport(writer); + } + + private void writeReport(Writer writer) throws IOException { + writeHeader(writer); + writer.flush(); + for (TaskReport tReport : driverTasks) + writeMetrics(writer, tReport); + writer.flush(); + } + + protected abstract void writeHeader(Writer writer) throws IOException; + + protected abstract void writeMetrics(Writer writer, TaskReport tReport) + throws IOException; +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractWorkerExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractWorkerExporter.java new file mode 100644 index 00000000..053a6d9b --- /dev/null +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractWorkerExporter.java @@ -0,0 +1,67 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.exporter; + +import java.io.IOException; +import java.io.Writer; +import java.util.ArrayList; +import java.util.List; + +import com.intel.cosbench.bench.Metrics; +import com.intel.cosbench.bench.Report; +import com.intel.cosbench.bench.TaskReport; +import com.intel.cosbench.model.DriverInfo; +import com.intel.cosbench.model.StageInfo; +import com.intel.cosbench.model.TaskInfo; +import com.intel.cosbench.model.WorkloadInfo; + +public abstract class AbstractWorkerExporter implements WorkerExporter { + + private StageInfo stageInfo; + + public StageInfo getStageInfo() { + return stageInfo; + } + + public void setStageInfo(StageInfo stageInfo) { + this.stageInfo = stageInfo; + } + + @Override + public void export(Writer writer) throws IOException { + writeReport(writer); + } + + private void writeReport(Writer writer) throws IOException { + writeHeader(writer); + writer.flush(); + for(TaskInfo taskInfo : stageInfo.getTaskInfos()){ + for(Metrics metrics: taskInfo.getWrReport()){ + writeMetrics(writer,metrics); + } + } + writer.flush(); + } + + protected abstract void writeHeader(Writer writer) throws IOException; + + protected abstract void writeMetrics(Writer writer, Metrics metrics) + throws IOException; + +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractWorkloadExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractWorkloadExporter.java index 9f48033f..1ed418c4 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractWorkloadExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/AbstractWorkloadExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -25,7 +25,7 @@ /** * This class is the base class for exporting workload information. - * + * * @author ywang19, qzheng7 * */ @@ -46,17 +46,17 @@ public void export(Writer writer) throws IOException { writeHeader(writer); writer.flush(); for (StageInfo stage : workload.getStageInfos()) { - for (Metrics metrics : stage.getReport()) + for (Metrics metrics : stage.getReport()) writeMetrics(writer, metrics, stage); - if (stage.getReport().getSize() == 0) - writeMetrics(writer, stage); + if (stage.getReport().getSize() == 0) + writeMetrics(writer, stage); } writer.flush(); } protected abstract void writeMetrics(Writer writer, StageInfo stage) throws IOException; - protected abstract void writeHeader(Writer writer) throws IOException; + protected abstract void writeHeader(Writer writer) throws IOException; protected abstract void writeMetrics(Writer writer, Metrics metrics, StageInfo stage) throws IOException; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVLatencyExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVLatencyExporter.java index 5e8814dc..f1b200f2 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVLatencyExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVLatencyExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -27,7 +27,7 @@ /** * This class is to export response time histogram data into CSV format. - * + * * @author ywang19, qzheng7 * */ @@ -38,7 +38,7 @@ protected void writeHeader(Writer writer) throws IOException { StringBuilder buffer = new StringBuilder(); buffer.append("ResTime").append(','); for (StageInfo stage : workload.getStageInfos()) { - writeOpType(buffer, stage); + writeOpType(buffer, stage); } buffer.setCharAt(buffer.length() - 1, '\n'); @@ -47,21 +47,21 @@ protected void writeHeader(Writer writer) throws IOException { private static void writeOpType(StringBuilder buffer, StageInfo stage) { for (Metrics metrics : stage.getReport()) { - String opt = metrics.getOpName(); - String spt = metrics.getSampleType(); - if (spt.equals(opt)){ /*just append normal stage*/ - String workName = null; - int workIdx = 1; - for (Work work : stage.getStage().getWorks()){ - if (work.getOperationIDs().contains(metrics.getOpId())) { - workName = "w" + workIdx + "-" + work.getName(); - break; - } - workIdx++; - } - buffer.append(stage.getId() + "-" + workName + "-" + opt); - buffer.append(',').append("(%)").append(','); - } + String opt = metrics.getOpName(); + String spt = metrics.getSampleType(); + if (spt.equals(opt)){ /*just append normal stage*/ + String workName = null; + int workIdx = 1; + for (Work work : stage.getStage().getWorks()){ + if (work.getOperationIDs().contains(metrics.getOpId())) { + workName = "w" + workIdx + "-" + work.getName(); + break; + } + workIdx++; + } + buffer.append(stage.getId() + "-" + workName + "-" + opt); + buffer.append(',').append("(%)").append(','); + } } } @@ -77,18 +77,18 @@ protected void writeHistogram(Writer writer, int idx) throws IOException { buffer.append(','); int metricsIdx = -1; for (StageInfo stage : workload.getStageInfos()) { - for (Metrics metrics : stage.getReport()) { - metricsIdx++; - if (!metrics.getOpName().equals(metrics.getSampleType()) || metrics.getLatency() == null) { - continue; /*skip for special work*/ - } - int count = metrics.getLatency().getHistoData()[idx]; - buffer.append(count).append(','); - accs[metricsIdx] += count; - double per = sums[metricsIdx] != 0 ? - accs[metricsIdx] / ((double) sums[metricsIdx]) : 0; - buffer.append(RATIO.format(per)).append(','); - } + for (Metrics metrics : stage.getReport()) { + metricsIdx++; + if (!metrics.getOpName().equals(metrics.getSampleType()) || metrics.getLatency() == null) { + continue; /*skip for special work*/ + } + int count = metrics.getLatency().getHistoData()[idx]; + buffer.append(count).append(','); + accs[metricsIdx] += count; + double per = sums[metricsIdx] != 0 ? + accs[metricsIdx] / ((double) sums[metricsIdx]) : 0; + buffer.append(RATIO.format(per)).append(','); + } } buffer.setCharAt(buffer.length() - 1, '\n'); writer.write(buffer.toString()); diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVMatrixExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVMatrixExporter.java index b920cb38..47404a45 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVMatrixExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVMatrixExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -27,7 +27,7 @@ /** * This class is to export performance matrix data into CSV format. - * + * * @author ywang19, qzheng7 * */ @@ -67,7 +67,7 @@ protected void writeMetrics(Writer writer, StageInfo stage, if (spt.equals(opt)) buffer.append(opt); else - buffer.append(opt + '-' + spt); + buffer.append(opt + '-' + spt); buffer.append(','); buffer.append(metrics.getSampleCount()).append(','); buffer.append(metrics.getByteCount()).append(','); @@ -81,9 +81,9 @@ protected void writeMetrics(Writer writer, StageInfo stage, double pt = r - metrics.getAvgXferTime(); if (pt > 0) - buffer.append(NUM.format(pt)); + buffer.append(NUM.format(pt)); else - buffer.append("N/A"); + buffer.append("N/A"); buffer.append(','); writeLatencyInfo(buffer, metrics.getLatency()); @@ -102,22 +102,22 @@ protected void writeMetrics(Writer writer, StageInfo stage, private static void writeLatencyInfo(StringBuilder buffer, Histogram latency) throws IOException { - if(latency == null) { - writePercentileRT(buffer, null); - writePercentileRT(buffer, null); - writePercentileRT(buffer, null); - writePercentileRT(buffer, null); - writePercentileRT(buffer, null); - writePercentileRT(buffer, null); - }else { - writePercentileRT(buffer, latency.get_60()); - writePercentileRT(buffer, latency.get_80()); - writePercentileRT(buffer, latency.get_90()); - writePercentileRT(buffer, latency.get_95()); - writePercentileRT(buffer, latency.get_99()); - writePercentileRT(buffer, latency.get_100()); - } - + if(latency == null) { + writePercentileRT(buffer, null); + writePercentileRT(buffer, null); + writePercentileRT(buffer, null); + writePercentileRT(buffer, null); + writePercentileRT(buffer, null); + writePercentileRT(buffer, null); + }else { + writePercentileRT(buffer, latency.get_60()); + writePercentileRT(buffer, latency.get_80()); + writePercentileRT(buffer, latency.get_90()); + writePercentileRT(buffer, latency.get_95()); + writePercentileRT(buffer, latency.get_99()); + writePercentileRT(buffer, latency.get_100()); + } + } private static void writePercentileRT(StringBuilder buffer, long[] resTime) { diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVRunExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVRunExporter.java index 52b51000..d5b0cd60 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVRunExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVRunExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -28,7 +28,7 @@ /** * This class is to export run information into CSV format. - * + * * @author ywang19, qzheng7 * */ @@ -64,11 +64,11 @@ protected void writeWorkload(Writer writer) throws IOException { buffer.setCharAt(buffer.length() - 1, '\n'); writer.write(buffer.toString()); } - + private static void appendDetailedState(StringBuilder buffer, WorkloadInfo workload) { for(StateInfo state : workload.getStateHistory()) { - String detailedState = state.getName().toLowerCase() + " @ " + DATETIME.format(state.getDate()); - buffer.append(detailedState).append(','); + String detailedState = state.getName().toLowerCase() + " @ " + DATETIME.format(state.getDate()); + buffer.append(detailedState).append(','); } } diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVStageExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVStageExporter.java index 60a89f25..d757ae60 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVStageExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVStageExporter.java @@ -1,128 +1,128 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.exporter; - -import static com.intel.cosbench.exporter.Formats.*; - -import java.io.*; -import java.util.Arrays; - -import org.apache.commons.lang.StringUtils; - -import com.intel.cosbench.bench.*; - -/** - * This class is to export stage information into CSV format. - * - * @author ywang19, qzheng7 - * - */ -class CSVStageExporter extends AbstractStageExporter { - - public CSVStageExporter() { - /* empty */ - } - - protected void writeHeader(Writer writer) throws IOException { - StringBuilder buffer = new StringBuilder(); - buffer.append("Timestamp").append(','); - char[] cs = new char[numOpTypes]; - Arrays.fill(cs, ','); - String suffix = new String(cs); - buffer.append("Op-Count").append(suffix); - buffer.append("Byte-Count").append(suffix); - buffer.append("Avg-ResTime").append(suffix); - buffer.append("Avg-ProcTime").append(suffix); - buffer.append("Throughput").append(suffix); - buffer.append("Bandwidth").append(suffix); - buffer.append("Succ-Ratio").append(suffix); - buffer.append("Version-Info"); - buffer.append(',').append(',').append('\n').append(','); - for (int i = 0; i < 7; i++) - // 7 metrics - for (Metrics metrics : snapshots[0].getReport()) - buffer.append( - StringUtils.join(new Object[] { - (metrics.getOpName().equals( - metrics.getSampleType()) ? null - : metrics.getOpName() + "-"), - metrics.getSampleType() })).append(','); - buffer.append("Min-Version").append(','); - buffer.append("Version").append(','); - buffer.append("Max-Version").append('\n'); - writer.write(buffer.toString()); - } - - protected void writeMetrics(Writer writer, Snapshot snapshot) - throws IOException { - StringBuilder buffer = new StringBuilder(); - buffer.append(TIME.format(snapshot.getTimestamp())).append(','); - Report report = snapshot.getReport(); - - if(report.getSize() == 0) - { - report.addMetrics(Metrics.newMetrics("na.na")); - } - - /* Operation Count */ - for (Metrics metrics : report) - buffer.append(metrics.getSampleCount()).append(','); - /* Byte Count */ - for (Metrics metrics : report) - buffer.append(metrics.getByteCount()).append(','); - /* Response Time */ - for (Metrics metrics : report) { - double r = metrics.getAvgResTime(); - if (r > 0) - buffer.append(NUM.format(r)); - else - buffer.append("N/A"); - buffer.append(','); - } - /* Transfer Time */ - for (Metrics metrics : report) { - double pt = metrics.getAvgResTime() - metrics.getAvgXferTime(); - if (pt > 0) - buffer.append(NUM.format(pt)); - else - buffer.append("N/A"); - buffer.append(','); - } - /* Throughput */ - for (Metrics metrics : report) - buffer.append(NUM.format(metrics.getThroughput())).append(','); - /* Bandwidth */ - for (Metrics metrics : report) - buffer.append(NUM.format(metrics.getBandwidth())).append(','); - /* Success Ratio */ - for (Metrics metrics : report) { - double t = (double) metrics.getRatio(); - if (t > 0) - buffer.append(RATIO.format(metrics.getRatio())); - else - buffer.append("N/A"); - buffer.append(','); - } - /* Version Info */ - buffer.append(snapshot.getMinVersion()).append(','); - buffer.append(snapshot.getVersion()).append(','); - buffer.append(snapshot.getMaxVersion()).append('\n'); - writer.write(buffer.toString()); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.exporter; + +import static com.intel.cosbench.exporter.Formats.*; + +import java.io.*; +import java.util.Arrays; + +import org.apache.commons.lang.StringUtils; + +import com.intel.cosbench.bench.*; + +/** + * This class is to export stage information into CSV format. + * + * @author ywang19, qzheng7 + * + */ +class CSVStageExporter extends AbstractStageExporter { + + public CSVStageExporter() { + /* empty */ + } + + protected void writeHeader(Writer writer) throws IOException { + StringBuilder buffer = new StringBuilder(); + buffer.append("Timestamp").append(','); + char[] cs = new char[numOpTypes]; + Arrays.fill(cs, ','); + String suffix = new String(cs); + buffer.append("Op-Count").append(suffix); + buffer.append("Byte-Count").append(suffix); + buffer.append("Avg-ResTime").append(suffix); + buffer.append("Avg-ProcTime").append(suffix); + buffer.append("Throughput").append(suffix); + buffer.append("Bandwidth").append(suffix); + buffer.append("Succ-Ratio").append(suffix); + buffer.append("Version-Info"); + buffer.append(',').append(',').append('\n').append(','); + for (int i = 0; i < 7; i++) + // 7 metrics + for (Metrics metrics : snapshots[0].getReport()) + buffer.append( + StringUtils.join(new Object[] { + (metrics.getOpName().equals( + metrics.getSampleType()) ? null + : metrics.getOpName() + "-"), + metrics.getSampleType() })).append(','); + buffer.append("Min-Version").append(','); + buffer.append("Version").append(','); + buffer.append("Max-Version").append('\n'); + writer.write(buffer.toString()); + } + + protected void writeMetrics(Writer writer, Snapshot snapshot) + throws IOException { + StringBuilder buffer = new StringBuilder(); + buffer.append(TIME.format(snapshot.getTimestamp())).append(','); + Report report = snapshot.getReport(); + + if(report.getSize() == 0) + { + report.addMetrics(Metrics.newMetrics("na.na")); + } + + /* Operation Count */ + for (Metrics metrics : report) + buffer.append(metrics.getSampleCount()).append(','); + /* Byte Count */ + for (Metrics metrics : report) + buffer.append(metrics.getByteCount()).append(','); + /* Response Time */ + for (Metrics metrics : report) { + double r = metrics.getAvgResTime(); + if (r > 0) + buffer.append(NUM.format(r)); + else + buffer.append("N/A"); + buffer.append(','); + } + /* Transfer Time */ + for (Metrics metrics : report) { + double pt = metrics.getAvgResTime() - metrics.getAvgXferTime(); + if (pt > 0) + buffer.append(NUM.format(pt)); + else + buffer.append("N/A"); + buffer.append(','); + } + /* Throughput */ + for (Metrics metrics : report) + buffer.append(NUM.format(metrics.getThroughput())).append(','); + /* Bandwidth */ + for (Metrics metrics : report) + buffer.append(NUM.format(metrics.getBandwidth())).append(','); + /* Success Ratio */ + for (Metrics metrics : report) { + double t = (double) metrics.getRatio(); + if (t > 0) + buffer.append(RATIO.format(metrics.getRatio())); + else + buffer.append("N/A"); + buffer.append(','); + } + /* Version Info */ + buffer.append(snapshot.getMinVersion()).append(','); + buffer.append(snapshot.getVersion()).append(','); + buffer.append(snapshot.getMaxVersion()).append('\n'); + writer.write(buffer.toString()); + } + +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVTaskExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVTaskExporter.java new file mode 100644 index 00000000..9930814b --- /dev/null +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVTaskExporter.java @@ -0,0 +1,106 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.exporter; + +import static com.intel.cosbench.exporter.Formats.NUM; +import static com.intel.cosbench.exporter.Formats.RATIO; +import static com.intel.cosbench.exporter.Formats.TIME; + +import java.io.IOException; +import java.io.Writer; +import java.util.Arrays; + +import org.apache.commons.lang.StringUtils; + +import com.intel.cosbench.bench.Metrics; +import com.intel.cosbench.bench.Report; +import com.intel.cosbench.bench.Snapshot; +import com.intel.cosbench.bench.TaskReport; + +public class CSVTaskExporter extends AbstractTaskExporter{ + + public CSVTaskExporter() { + /* empty */ + } + protected void writeHeader(Writer writer) throws IOException { + StringBuilder buffer = new StringBuilder(); + char[] cs = new char[8]; + buffer.append("Op-Type").append(','); + buffer.append("Sample-Type").append(','); + buffer.append("Op-Count").append(','); + buffer.append("Byte-Count").append(','); + buffer.append("Avg-ResTime").append(','); + buffer.append("Avg-ProcTime").append(','); + buffer.append("Throughput").append(','); + buffer.append("Bandwidth").append(','); + buffer.append("Succ-Ratio").append('\n'); + writer.write(buffer.toString()); + } + + protected void writeMetrics(Writer writer,TaskReport tReport)throws IOException { + StringBuilder buffer = new StringBuilder(); + Report report = tReport.getReport(); + /*Operation Type*/ + for(Metrics metrics:report) + buffer.append(metrics.getOpType()).append(','); + /*sample Type*/ + for(Metrics metrics:report) + buffer.append(metrics.getSampleType()).append(','); + /* Operation Count */ + for (Metrics metrics :report) + buffer.append(metrics.getSampleCount()).append(','); + /* Byte Count */ + for (Metrics metrics : report) + buffer.append(metrics.getByteCount()).append(','); + /* Response Time */ + for (Metrics metrics : report) { + double r = metrics.getAvgResTime(); + if (r > 0) + buffer.append(NUM.format(r)); + else + buffer.append("N/A"); + buffer.append(','); + } + /* Transfer Time */ + for (Metrics metrics : report) { + double pt = metrics.getAvgResTime() - metrics.getAvgXferTime(); + if (pt > 0) + buffer.append(NUM.format(pt)); + else + buffer.append("N/A"); + buffer.append(','); + } + /* Throughput */ + for (Metrics metrics : report) + buffer.append(NUM.format(metrics.getThroughput())).append(','); + /* Bandwidth */ + for (Metrics metrics : report) + buffer.append(NUM.format(metrics.getBandwidth())).append(','); + /* Success Ratio */ + for (Metrics metrics : report) { + double t = (double) metrics.getRatio(); + if (t > 0) + buffer.append(RATIO.format(metrics.getRatio())); + else + buffer.append("N/A"); + buffer.append('\n'); + } + writer.write(buffer.toString()); + } +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVWorkerExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVWorkerExporter.java new file mode 100644 index 00000000..73906ee9 --- /dev/null +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVWorkerExporter.java @@ -0,0 +1,90 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.exporter; + +import static com.intel.cosbench.exporter.Formats.NUM; +import static com.intel.cosbench.exporter.Formats.RATIO; + +import java.io.IOException; +import java.io.Writer; + +import com.intel.cosbench.bench.Metrics; +import com.intel.cosbench.bench.Report; +import com.intel.cosbench.bench.TaskReport; + +public class CSVWorkerExporter extends AbstractWorkerExporter{ + + @Override + protected void writeHeader(Writer writer) throws IOException { + StringBuilder buffer = new StringBuilder(); + char[] cs = new char[8]; + buffer.append("Op-Type").append(','); + buffer.append("Sample-Type").append(','); + buffer.append("Op-Count").append(','); + buffer.append("Byte-Count").append(','); + buffer.append("Avg-ResTime").append(','); + buffer.append("Avg-ProcTime").append(','); + buffer.append("Throughput").append(','); + buffer.append("Bandwidth").append(','); + buffer.append("Succ-Ratio").append('\n'); + writer.write(buffer.toString()); + } + + @Override + protected void writeMetrics(Writer writer,Metrics metrics)throws IOException { + StringBuilder buffer = new StringBuilder(); + /*Operation Type*/ + buffer.append(metrics.getOpType()).append(','); + /*sample Type*/ + buffer.append(metrics.getSampleType()).append(','); + /* Operation Count */ + buffer.append(metrics.getSampleCount()).append(','); + /* Byte Count */ + buffer.append(metrics.getByteCount()).append(','); + /* Response Time */ + double r = metrics.getAvgResTime(); + if (r > 0) + buffer.append(NUM.format(r)); + else + buffer.append("N/A"); + buffer.append(','); + + /* Transfer Time */ + double pt = metrics.getAvgResTime() - metrics.getAvgXferTime(); + if (pt > 0) + buffer.append(NUM.format(pt)); + else + buffer.append("N/A"); + buffer.append(','); + + /* Throughput */ + buffer.append(NUM.format(metrics.getThroughput())).append(','); + /* Bandwidth */ + buffer.append(NUM.format(metrics.getBandwidth())).append(','); + /* Success Ratio */ + double t = (double) metrics.getRatio(); + if (t > 0) + buffer.append(RATIO.format(metrics.getRatio())); + else + buffer.append("N/A"); + buffer.append('\n'); + writer.write(buffer.toString()); + } + +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVWorkloadExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVWorkloadExporter.java index 9f0f911d..9f6b5e21 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVWorkloadExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/CSVWorkloadExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -27,7 +27,7 @@ /** * This class is to export workload information into CSV format. - * + * * @author ywang19, qzheng7 * */ @@ -81,9 +81,9 @@ protected void writeMetrics(Writer writer, Metrics metrics, StageInfo stage) double pt = r - metrics.getAvgXferTime(); if (pt > 0) - buffer.append(NUM.format(pt)); + buffer.append(NUM.format(pt)); else - buffer.append("N/A"); + buffer.append("N/A"); buffer.append(','); writeLatencyInfo(buffer, metrics.getLatency()); @@ -96,18 +96,18 @@ protected void writeMetrics(Writer writer, Metrics metrics, StageInfo stage) buffer.append("N/A").append(','); buffer.append(stage.getState().name().toLowerCase()).append(','); for (StateInfo state : stage.getStateHistory()) { - buffer.append( - state.getName().toLowerCase() + " @ " - + DATETIME.format(state.getDate())).append(','); + buffer.append( + state.getName().toLowerCase() + " @ " + + DATETIME.format(state.getDate())).append(','); } buffer.setCharAt(buffer.length() - 1, '\n'); writer.write(buffer.toString()); } - + @Override protected void writeMetrics(Writer writer, StageInfo stage) throws IOException { - StringBuilder buffer = new StringBuilder(); + StringBuilder buffer = new StringBuilder(); buffer.append(stage.getStage().getName()).append(','); buffer.append("N/A").append(','); buffer.append("N/A").append(','); @@ -126,9 +126,9 @@ protected void writeMetrics(Writer writer, StageInfo stage) buffer.append("N/A").append(','); buffer.append(stage.getState().name().toLowerCase()).append(','); for (StateInfo state : stage.getStateHistory()) { - buffer.append( - state.getName().toLowerCase() + " @ " - + DATETIME.format(state.getDate())).append(','); + buffer.append( + state.getName().toLowerCase() + " @ " + + DATETIME.format(state.getDate())).append(','); } buffer.setCharAt(buffer.length() - 1, '\n'); writer.write(buffer.toString()); @@ -136,21 +136,21 @@ protected void writeMetrics(Writer writer, StageInfo stage) private static void writeLatencyInfo(StringBuilder buffer, Histogram latency) throws IOException { - if(latency == null) { + if(latency == null) { writePercentileRT(buffer,null); writePercentileRT(buffer, null); writePercentileRT(buffer, null); writePercentileRT(buffer, null); writePercentileRT(buffer, null); writePercentileRT(buffer, null); - }else { - writePercentileRT(buffer, latency.get_60()); - writePercentileRT(buffer, latency.get_80()); - writePercentileRT(buffer, latency.get_90()); - writePercentileRT(buffer, latency.get_95()); - writePercentileRT(buffer, latency.get_99()); - writePercentileRT(buffer, latency.get_100()); - } + }else { + writePercentileRT(buffer, latency.get_60()); + writePercentileRT(buffer, latency.get_80()); + writePercentileRT(buffer, latency.get_90()); + writePercentileRT(buffer, latency.get_95()); + writePercentileRT(buffer, latency.get_99()); + writePercentileRT(buffer, latency.get_100()); + } } private static void writePercentileRT(StringBuilder buffer, long[] resTime) { diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/ExportException.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/ExportException.java index ede8ce8a..09cf468a 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/ExportException.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/ExportException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/Exporters.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/Exporters.java index 1aa86dce..d0056196 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/Exporters.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/Exporters.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,21 +12,35 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; +import com.intel.cosbench.bench.TaskReport; import com.intel.cosbench.model.*; /** * This class is a wrapper to construct different exporters by providing static construction methods. - * + * * @author ywang19, qzheng7 * */ public class Exporters { + public static WorkerExporter newWorkExporter(StageInfo info){ + AbstractWorkerExporter exporter = new CSVWorkerExporter(); + exporter.setStageInfo(info); + return exporter; + } + + public static TaskExporter newTaskExporter(WorkloadInfo workloadInfo,DriverInfo driverInfo){ + AbstractTaskExporter exporter = new CSVTaskExporter(); + exporter.setWorkloadInfo(workloadInfo); + exporter.setDriverInfo(driverInfo); + return exporter; + } + public static RunExporter newRunExporter(WorkloadInfo workload) { AbstractRunExporter exporter = new CSVRunExporter(); exporter.setWorkload(workload); @@ -56,7 +70,7 @@ public static LogExporter newLogExporter(WorkloadInfo workload) { exporter.setWorkload(workload); return exporter; } - + public static LogExporter newScriptLogExporter(WorkloadInfo workload) { ScriptsLogExporter exporter = new ScriptsLogExporter(); exporter.setWorkload(workload); @@ -69,4 +83,6 @@ public static MatrixExporter newMatrixExporter(WorkloadInfo workload) { return exporter; } + + } diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/Formats.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/Formats.java index 0bd9fa66..bad2c10b 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/Formats.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/Formats.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -21,7 +21,7 @@ /** * This class defines necessary data/number formats. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/LatencyExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/LatencyExporter.java index 30b7724f..0f6afc2e 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/LatencyExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/LatencyExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/LogExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/LogExporter.java index 02320a89..f8bedb55 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/LogExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/LogExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/MatrixExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/MatrixExporter.java index 4c1ee511..17f5689a 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/MatrixExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/MatrixExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/RunExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/RunExporter.java index 41d5b187..e3da32f5 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/RunExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/RunExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/ScriptsLogExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/ScriptsLogExporter.java index 12e5c898..1433511e 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/ScriptsLogExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/ScriptsLogExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; @@ -41,12 +41,12 @@ public void setWorkload(WorkloadInfo workload) { @Override public void export(Writer writer) throws IOException { - exportWorkloadLog(writer); - for (StageInfo stage : workload.getStageInfos()) + exportWorkloadLog(writer); + for (StageInfo stage : workload.getStageInfos()) exportStageLog(writer, stage); writer.flush(); } - + private void exportWorkloadLog(Writer writer) throws IOException { writer.write("========================="); writer.write("========================="); @@ -57,16 +57,16 @@ private void exportWorkloadLog(Writer writer) throws IOException { String wsId = workload.getId(); doExportLog(writer, wsId); writer.flush(); - } - + } + private void doExportLog(Writer writer, String wsId) throws IOException { for (DriverInfo driver : workload.getDriverInfos()) { - Map logMap = driver.getLogMap(); - if (logMap.containsKey(wsId)) { - exportScriptLog(writer, logMap.get(wsId), driver.getName()); - } + Map logMap = driver.getLogMap(); + if (logMap.containsKey(wsId)) { + exportScriptLog(writer, logMap.get(wsId), driver.getName()); + } } - } + } private void exportStageLog(Writer writer, StageInfo stage) throws IOException { writer.write("========================="); @@ -79,20 +79,20 @@ private void exportStageLog(Writer writer, StageInfo stage) throws IOException { doExportLog(writer, wsId); writer.flush(); } - + private void exportScriptLog(Writer writer, String logCtx, String driver) throws IOException { - int idx = StringUtils.indexOf(logCtx, ";"); - if (idx < 0 || idx+1 == logCtx.length()) - return; - String scriptName = StringUtils.left(logCtx, idx); - String log = StringUtils.substring(logCtx, idx+1); + int idx = StringUtils.indexOf(logCtx, ";"); + if (idx < 0 || idx+1 == logCtx.length()) + return; + String scriptName = StringUtils.left(logCtx, idx); + String log = StringUtils.substring(logCtx, idx+1); writer.write("-----------------"); writer.write("-----------------"); writer.write(" driver: " + driver + " script: " + scriptName + ' '); writer.write("-----------------"); writer.write("-----------------"); writer.write('\n'); - writer.write(log); - } + writer.write(log); + } } diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/SimpleLogExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/SimpleLogExporter.java index e55c112e..6af0c514 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/SimpleLogExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/SimpleLogExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/StageExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/StageExporter.java index 64cc9df3..4b9a8f28 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/StageExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/StageExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/TaskExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/TaskExporter.java new file mode 100644 index 00000000..1798cd3a --- /dev/null +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/TaskExporter.java @@ -0,0 +1,26 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.exporter; + +import java.io.IOException; +import java.io.Writer; + +public interface TaskExporter { + public void export(Writer writer) throws IOException; +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/WorkerExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/WorkerExporter.java new file mode 100644 index 00000000..d7d04476 --- /dev/null +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/WorkerExporter.java @@ -0,0 +1,26 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.intel.cosbench.exporter; + +import java.io.IOException; +import java.io.Writer; + +public interface WorkerExporter { + public void export(Writer writer) throws IOException; +} diff --git a/dev/cosbench-core/src/com/intel/cosbench/exporter/WorkloadExporter.java b/dev/cosbench-core/src/com/intel/cosbench/exporter/WorkloadExporter.java index 06da171d..98a6a79e 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/exporter/WorkloadExporter.java +++ b/dev/cosbench-core/src/com/intel/cosbench/exporter/WorkloadExporter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.exporter; diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/ControllerInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/ControllerInfo.java index de06b625..f099b1b5 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/ControllerInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/ControllerInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; @@ -27,4 +27,5 @@ public interface ControllerInfo { public DriverInfo[] getDriverInfos(); + public String getVersion(); } diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/DriverInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/DriverInfo.java index c0d3b554..67ed2c47 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/DriverInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/DriverInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; @@ -26,9 +26,10 @@ public interface DriverInfo { public String getUrl(); public void setAliveState(boolean aliveState); - + public boolean getAliveState(); - + public Map getLogMap(); - + + public String getVersion(); } diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/LifeCycle.java b/dev/cosbench-core/src/com/intel/cosbench/model/LifeCycle.java index 416d8ab9..339e583d 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/LifeCycle.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/LifeCycle.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/MissionInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/MissionInfo.java index 76cffe2e..db5a9f2e 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/MissionInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/MissionInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,15 +12,17 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; import java.util.Date; +import java.util.List; import com.intel.cosbench.bench.Benchmark; import com.intel.cosbench.bench.ErrorStatistics; +import com.intel.cosbench.bench.Report; import com.intel.cosbench.config.Mission; import com.intel.cosbench.log.LogManager; @@ -41,7 +43,9 @@ public interface MissionInfo extends LifeCycle, Benchmark { public int getWorkerCount(); public WorkerInfo[] getWorkerInfos(); - + public ErrorStatistics getErrorStatistics(); + public List getWorkerReports(); + } diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/MissionState.java b/dev/cosbench-core/src/com/intel/cosbench/model/MissionState.java index cfd8e7a4..48601806 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/MissionState.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/MissionState.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; @@ -30,7 +30,7 @@ public enum MissionState { LAUNCHED, FINISHED, - + FAILED, ACCOMPLISHED, diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/ScheduleInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/ScheduleInfo.java index 39f51f00..5ed5de20 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/ScheduleInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/ScheduleInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/StageInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/StageInfo.java index ad022bf7..c0e74523 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/StageInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/StageInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,12 +12,13 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; import java.util.Date; +import java.util.List; import java.util.Set; import com.intel.cosbench.bench.*; @@ -29,9 +30,9 @@ public interface StageInfo extends LifeCycle, Benchmark { public String getId(); public StageState getState(); - + public void setState(String state, Date date); - + public void setState(StageState state, boolean archived); public StateInfo[] getStateHistory(); @@ -51,11 +52,15 @@ public interface StageInfo extends LifeCycle, Benchmark { public TaskInfo[] getTaskInfos(); public Snapshot[] getSnapshots(); - + public int getSnapshotCount(); - public void setReport(Report report); - - public ListRegistry getSnapshotRegistry(); + public void setReport(Report report); + + public ListRegistry getSnapshotRegistry(); + + public List getTaskReports(); + + public void setTaskReports(List taskReports); } diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/StageState.java b/dev/cosbench-core/src/com/intel/cosbench/model/StageState.java index b80515f0..77da7221 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/StageState.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/StageState.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; @@ -22,7 +22,7 @@ public enum StageState { WAITING, - + SLEEPING, BOOTING, @@ -38,7 +38,7 @@ public enum StageState { CLOSING, COMPLETED, - + FAILED, TERMINATED, diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/StateInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/StateInfo.java index 653d80fd..8947b72a 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/StateInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/StateInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/StateRegistry.java b/dev/cosbench-core/src/com/intel/cosbench/model/StateRegistry.java index ccbf69c6..8ee34744 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/StateRegistry.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/StateRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; @@ -48,9 +48,9 @@ public Date getDate() { public void addState(String name) { addItem(new StateItem(name, new Date())); } - + public void addState(String name, Date date) { - addItem(new StateItem(name, date)); + addItem(new StateItem(name, date)); } public StateInfo getState(int index) { diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/TaskInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/TaskInfo.java index 520d8e82..986f4b50 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/TaskInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/TaskInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,12 +12,15 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; +import java.util.ArrayList; + import com.intel.cosbench.bench.Benchmark; +import com.intel.cosbench.bench.Metrics; public interface TaskInfo extends LifeCycle, Benchmark { @@ -31,4 +34,6 @@ public interface TaskInfo extends LifeCycle, Benchmark { public String getLog(); + public ArrayList getWrReport(); + } diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/TaskState.java b/dev/cosbench-core/src/com/intel/cosbench/model/TaskState.java index 558bb339..a433d2ff 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/TaskState.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/TaskState.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; @@ -32,7 +32,7 @@ public enum TaskState { LAUNCHED, FINISHED, - + FAILED, ACCOMPLISHED, diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/WorkerInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/WorkerInfo.java index 0674e152..2700f941 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/WorkerInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/WorkerInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/WorkloadInfo.java b/dev/cosbench-core/src/com/intel/cosbench/model/WorkloadInfo.java index 33e598fb..3d9c6512 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/WorkloadInfo.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/WorkloadInfo.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; @@ -52,12 +52,12 @@ public interface WorkloadInfo extends LifeCycle, Benchmark { public int getSnapshotCount(); - public void setWorkload(Workload workload); - - public void setArchived(boolean archived); - - public boolean getArchived(); - - public DriverInfo[] getDriverInfos(); + public void setWorkload(Workload workload); + + public void setArchived(boolean archived); + + public boolean getArchived(); + + public DriverInfo[] getDriverInfos(); } diff --git a/dev/cosbench-core/src/com/intel/cosbench/model/WorkloadState.java b/dev/cosbench-core/src/com/intel/cosbench/model/WorkloadState.java index 19b04054..fe4e77d6 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/model/WorkloadState.java +++ b/dev/cosbench-core/src/com/intel/cosbench/model/WorkloadState.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.model; @@ -26,7 +26,7 @@ public enum WorkloadState { PROCESSING, FINISHED, - + FAILED, TERMINATED, diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/AbortResponse.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/AbortResponse.java index a74e141b..70325bc6 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/AbortResponse.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/AbortResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; @@ -24,7 +24,7 @@ /** * The response to get log from driver when aborted. - * + * * @author ywang19, qzheng7 * */ @@ -44,7 +44,7 @@ public String getDriverLog() { public void setDriverLog(String driverLog) { this.driverLog = driverLog; } - + public List getReport() { return report; } diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/CloseResponse.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/CloseResponse.java index fb9b0694..affe1a6c 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/CloseResponse.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/CloseResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,20 +12,22 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; +import java.util.ArrayList; import java.util.HashMap; import java.util.List; import com.intel.cosbench.bench.Metrics; +import com.intel.cosbench.bench.Report; import com.intel.cosbench.model.TaskState; /** * The response to get log from driver when closed. - * + * * @author ywang19, qzheng7 * */ @@ -35,17 +37,26 @@ public class CloseResponse extends Response { private String driverLog; /* driver log */ private TaskState state; private HashMap errorStatistics; /* mission error statistics*/ + private List wrReport; + + public List getWrReport() { + return wrReport; + } + + public void setWrReport(List wrReport) { + this.wrReport = wrReport; + } public CloseResponse() { /* empty */ } - + public TaskState getState(){ - return state; + return state; } - + public void setState(TaskState state){ - this.state = state; + this.state = state; } public List getReport() { @@ -64,12 +75,12 @@ public void setDriverLog(String log) { this.driverLog = log; } - public HashMap getErrorStatistics() { - return errorStatistics; - } + public HashMap getErrorStatistics() { + return errorStatistics; + } + + public void setErrorStatistics(HashMap errorStatistics) { + this.errorStatistics = errorStatistics; + } - public void setErrorStatistics(HashMap errorStatistics) { - this.errorStatistics = errorStatistics; - } - } diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/LaunchResponse.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/LaunchResponse.java index 0f6167b8..2571046e 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/LaunchResponse.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/LaunchResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; /** * The response to launch workload from driver. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/LoginResponse.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/LoginResponse.java index 2404ad7d..2a47587a 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/LoginResponse.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/LoginResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; /** * The response to login. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/PingResponse.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/PingResponse.java index 608b1019..fa0f17f2 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/PingResponse.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/PingResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; /** * The response for request to ping drivers. - * + * * @author ywang19, qzheng7 * */ @@ -29,7 +29,7 @@ public class PingResponse extends Response { private String address; /* driver address */ private String timeStamp; /* driver current time */ - public PingResponse() { + public PingResponse() { /* empty */ } @@ -48,13 +48,13 @@ public String getAddress() { public void setAddress(String address) { this.address = address; } - + public String getTimeStamp() { - return timeStamp; - } + return timeStamp; + } - public void setTimeStamp(String timeStamp) { - this.timeStamp = timeStamp; - } + public void setTimeStamp(String timeStamp) { + this.timeStamp = timeStamp; + } } diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/QueryResponse.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/QueryResponse.java index a7ea7285..3518dccc 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/QueryResponse.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/QueryResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; @@ -24,7 +24,7 @@ /** * The response for request to query snapshot from driver. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/Response.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/Response.java index 0762977f..a023c750 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/Response.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/Response.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; /** * The base class encapsulates HTTP response from Drivers. - * + * * @author ywang19, qzheng7 * */ @@ -38,7 +38,7 @@ public Response(int code) { this.code = code; this.succ = false; } - + public Response(int code, String error) { this.code = code; this.succ = false; diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/SubmitResponse.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/SubmitResponse.java index 5cffcf39..a962a4be 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/SubmitResponse.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/SubmitResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; /** * The response for request to sumit workload to drivers. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/protocol/TriggerResponse.java b/dev/cosbench-core/src/com/intel/cosbench/protocol/TriggerResponse.java index 83ac072c..23ed8eb8 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/protocol/TriggerResponse.java +++ b/dev/cosbench-core/src/com/intel/cosbench/protocol/TriggerResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,34 +12,34 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.protocol; public class TriggerResponse extends Response{ - private String PID; - private String scriptLog; + private String PID; + private String scriptLog; - public TriggerResponse() { - /*empty*/ - } + public TriggerResponse() { + /*empty*/ + } - public String getPID() { - return PID; - } + public String getPID() { + return PID; + } - public void setPID(String pID) { - this.PID = pID; - } + public void setPID(String pID) { + this.PID = pID; + } - public String getScriptLog() { - return scriptLog; - } + public String getScriptLog() { + return scriptLog; + } - public void setScriptLog(String scriptLog) { - this.scriptLog = scriptLog; - } + public void setScriptLog(String scriptLog) { + this.scriptLog = scriptLog; + } } diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/AbortedException.java b/dev/cosbench-core/src/com/intel/cosbench/service/AbortedException.java index 7d9dc8af..8091787f 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/AbortedException.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/AbortedException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/AbstractServiceFactory.java b/dev/cosbench-core/src/com/intel/cosbench/service/AbstractServiceFactory.java index 1752b300..297c518f 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/AbstractServiceFactory.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/AbstractServiceFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; @@ -25,7 +25,7 @@ /** * The class abstracts basic methods need for cosbench service. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/CancelledException.java b/dev/cosbench-core/src/com/intel/cosbench/service/CancelledException.java index 74e0ce3b..6e198731 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/CancelledException.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/CancelledException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; @@ -21,7 +21,7 @@ /** * This exception is used to adapt an interrupted exception that often * occurs when a user is canceling a task - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/ControllerService.java b/dev/cosbench-core/src/com/intel/cosbench/service/ControllerService.java index 336b22f5..bcfc94c9 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/ControllerService.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/ControllerService.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; @@ -25,18 +25,18 @@ /** * The interface for controller service. - * + * * @author ywang19, qzheng7 * */ public interface ControllerService { public String submit(XmlConfig config); - - public String resubmit(String id) throws IOException; + + public String resubmit(String id) throws IOException; public void fire(String id); - + public boolean changeOrder(String id, String neighId, boolean up); public void cancel(String id); @@ -48,17 +48,17 @@ public interface ControllerService { public WorkloadInfo[] getActiveWorkloads(); public WorkloadInfo[] getHistoryWorkloads(); - + public WorkloadInfo[] getArchivedWorkloads(); public File getWorkloadLog(WorkloadInfo info); public File getWorkloadConfig(WorkloadInfo info); - + public WorkloadLoader getWorkloadLoader(); - + public boolean getloadArch(); - + public void setloadArch(boolean loadArch); } diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/ControllerServiceFactory.java b/dev/cosbench-core/src/com/intel/cosbench/service/ControllerServiceFactory.java index 195efe56..e9d25c27 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/ControllerServiceFactory.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/ControllerServiceFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/DriverService.java b/dev/cosbench-core/src/com/intel/cosbench/service/DriverService.java index 6ceddc4d..3ae6d33b 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/DriverService.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/DriverService.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; @@ -23,7 +23,7 @@ /** * The interface for driver service. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/DriverServiceFactory.java b/dev/cosbench-core/src/com/intel/cosbench/service/DriverServiceFactory.java index 61267a54..9b748974 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/DriverServiceFactory.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/DriverServiceFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/IllegalStateException.java b/dev/cosbench-core/src/com/intel/cosbench/service/IllegalStateException.java index fbacc291..03853288 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/IllegalStateException.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/IllegalStateException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/TimeoutException.java b/dev/cosbench-core/src/com/intel/cosbench/service/TimeoutException.java index 8c6d3489..0e003006 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/TimeoutException.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/TimeoutException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/UnexpectedException.java b/dev/cosbench-core/src/com/intel/cosbench/service/UnexpectedException.java index 40d1a05b..a418cbbf 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/UnexpectedException.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/UnexpectedException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.service; @@ -23,9 +23,9 @@ * elegant manner, while normally they will eventually be caught and LOGGED in a * top level place. Developers should consult logs for details of these * exceptions. - * + * * @author ywang19, qzheng7 - * + * */ public class UnexpectedException extends RuntimeException { diff --git a/dev/cosbench-core/src/com/intel/cosbench/service/WorkloadLoader.java b/dev/cosbench-core/src/com/intel/cosbench/service/WorkloadLoader.java index aa0aed36..d751045f 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/service/WorkloadLoader.java +++ b/dev/cosbench-core/src/com/intel/cosbench/service/WorkloadLoader.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.service; import java.io.IOException; @@ -7,9 +25,9 @@ public interface WorkloadLoader { - public List loadWorkloadRun() throws IOException; + public List loadWorkloadRun() throws IOException; + + public void loadWorkloadPageInfo(WorkloadInfo workloadContext) throws IOException; - public void loadWorkloadPageInfo(WorkloadInfo workloadContext) throws IOException; - - public void loadStagePageInfo(WorkloadInfo workloadContext, String stageId) throws IOException; + public void loadStagePageInfo(WorkloadInfo workloadContext, String stageId) throws IOException; } diff --git a/dev/cosbench-core/src/com/intel/cosbench/utils/AuthValidator.java b/dev/cosbench-core/src/com/intel/cosbench/utils/AuthValidator.java index 91bec878..5941f1e3 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/utils/AuthValidator.java +++ b/dev/cosbench-core/src/com/intel/cosbench/utils/AuthValidator.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.utils; import static org.apache.commons.lang.SystemUtils.IS_OS_WINDOWS; @@ -16,58 +34,58 @@ public class AuthValidator { - private static final String CFG_FILE_KEY = "cosbench.web.cosbenchUsers"; - - private static final String UNIX_DEFAULT_CFG_FILE = "/etc/cosbench/cosbench-users.xml"; - - private static final String WIN_DEFAULT_CFG_FILE = "C:\\cosbench-users.xml"; - - public static final String USERNAME = "anonymous"; - public static final String PASSWD = "cosbench"; - - public AuthValidator() { - /* empty */ - } - - public static boolean NeedLogon() throws Exception{ - Map userInfo = new HashMap(); - return NeedLogon(userInfo); - } - - public static boolean NeedLogon(Map userInfo) throws Exception { - - File file = new File(getConfigFile()); - DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - DocumentBuilder db = dbf.newDocumentBuilder(); - Document doc = db.parse(file); - doc.getDocumentElement().normalize(); - - NodeList userNodeLst = doc.getElementsByTagName("user"); - - for (int s = 0; s < userNodeLst.getLength(); s++) { - Node userNode = userNodeLst.item(s); - if (userNode.getNodeType() == Node.ELEMENT_NODE) { - Element Elmnt = (Element) userNode; - String username = Elmnt.getAttribute("username").toString(); - String password = Elmnt.getAttribute("password").toString(); - if (username.equals(USERNAME) && password.equals(PASSWD)) - return true; - - if (userInfo != null & !userInfo.containsKey(username)) - userInfo.put(username, password); - } - } - return false; - } - - public static String getConfigFile() { - String configFile; - if ((configFile = System.getProperty(CFG_FILE_KEY)) != null) - return configFile; - if (new File("cosbench-users.xml").exists()) - return "cosbench-users.xml"; - if (new File("conf/cosbench-users.xml").exists()) - return "conf/cosbench-users.xml"; - return IS_OS_WINDOWS ? WIN_DEFAULT_CFG_FILE : UNIX_DEFAULT_CFG_FILE; - } + private static final String CFG_FILE_KEY = "cosbench.web.cosbenchUsers"; + + private static final String UNIX_DEFAULT_CFG_FILE = "/etc/cosbench/cosbench-users.xml"; + + private static final String WIN_DEFAULT_CFG_FILE = "C:\\cosbench-users.xml"; + + public static final String USERNAME = "anonymous"; + public static final String PASSWD = "cosbench"; + + public AuthValidator() { + /* empty */ + } + + public static boolean NeedLogon() throws Exception{ + Map userInfo = new HashMap(); + return NeedLogon(userInfo); + } + + public static boolean NeedLogon(Map userInfo) throws Exception { + + File file = new File(getConfigFile()); + DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + DocumentBuilder db = dbf.newDocumentBuilder(); + Document doc = db.parse(file); + doc.getDocumentElement().normalize(); + + NodeList userNodeLst = doc.getElementsByTagName("user"); + + for (int s = 0; s < userNodeLst.getLength(); s++) { + Node userNode = userNodeLst.item(s); + if (userNode.getNodeType() == Node.ELEMENT_NODE) { + Element Elmnt = (Element) userNode; + String username = Elmnt.getAttribute("username").toString(); + String password = Elmnt.getAttribute("password").toString(); + if (username.equals(USERNAME) && password.equals(PASSWD)) + return true; + + if (userInfo != null & !userInfo.containsKey(username)) + userInfo.put(username, password); + } + } + return false; + } + + public static String getConfigFile() { + String configFile; + if ((configFile = System.getProperty(CFG_FILE_KEY)) != null) + return configFile; + if (new File("cosbench-users.xml").exists()) + return "cosbench-users.xml"; + if (new File("conf/cosbench-users.xml").exists()) + return "conf/cosbench-users.xml"; + return IS_OS_WINDOWS ? WIN_DEFAULT_CFG_FILE : UNIX_DEFAULT_CFG_FILE; + } } diff --git a/dev/cosbench-core/src/com/intel/cosbench/utils/ListRegistry.java b/dev/cosbench-core/src/com/intel/cosbench/utils/ListRegistry.java index d3f0bb14..25c92f8f 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/utils/ListRegistry.java +++ b/dev/cosbench-core/src/com/intel/cosbench/utils/ListRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.utils; diff --git a/dev/cosbench-core/src/com/intel/cosbench/utils/MapRegistry.java b/dev/cosbench-core/src/com/intel/cosbench/utils/MapRegistry.java index f822f5b5..98ec16fd 100644 --- a/dev/cosbench-core/src/com/intel/cosbench/utils/MapRegistry.java +++ b/dev/cosbench-core/src/com/intel/cosbench/utils/MapRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.utils; diff --git a/dev/cosbench-driver-web/META-INF/MANIFEST.MF b/dev/cosbench-driver-web/META-INF/MANIFEST.MF index b5d9dfbb..2fd96cc6 100644 --- a/dev/cosbench-driver-web/META-INF/MANIFEST.MF +++ b/dev/cosbench-driver-web/META-INF/MANIFEST.MF @@ -3,7 +3,7 @@ Web-ContextPath: driver Bundle-ManifestVersion: 2 Bundle-Name: COSBench Driver Web Bundle Bundle-SymbolicName: cosbench-driver-web -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.bench, diff --git a/dev/cosbench-driver-web/WEB-INF/freemarker/400.ftl b/dev/cosbench-driver-web/WEB-INF/freemarker/400.ftl index 94213dee..3cc9448d 100644 --- a/dev/cosbench-driver-web/WEB-INF/freemarker/400.ftl +++ b/dev/cosbench-driver-web/WEB-INF/freemarker/400.ftl @@ -6,7 +6,7 @@ 400 Bad Request -<#include "header.ftl"> +<#include "head.ftl">

diff --git a/dev/cosbench-driver-web/WEB-INF/freemarker/404.ftl b/dev/cosbench-driver-web/WEB-INF/freemarker/404.ftl index 2607ac21..61dfeb5c 100644 --- a/dev/cosbench-driver-web/WEB-INF/freemarker/404.ftl +++ b/dev/cosbench-driver-web/WEB-INF/freemarker/404.ftl @@ -6,7 +6,7 @@ 404 Not Found -<#include "header.ftl"> +<#include "head.ftl">

diff --git a/dev/cosbench-driver-web/WEB-INF/freemarker/500.ftl b/dev/cosbench-driver-web/WEB-INF/freemarker/500.ftl index 6647775e..8e732cf0 100644 --- a/dev/cosbench-driver-web/WEB-INF/freemarker/500.ftl +++ b/dev/cosbench-driver-web/WEB-INF/freemarker/500.ftl @@ -6,7 +6,7 @@ Internal Error -<#include "header.ftl"> +<#include "head.ftl">

diff --git a/dev/cosbench-driver-web/WEB-INF/freemarker/head.ftl b/dev/cosbench-driver-web/WEB-INF/freemarker/head.ftl new file mode 100644 index 00000000..c745d920 --- /dev/null +++ b/dev/cosbench-driver-web/WEB-INF/freemarker/head.ftl @@ -0,0 +1,5 @@ + <#-- end of header --> diff --git a/dev/cosbench-driver-web/WEB-INF/freemarker/header.ftl b/dev/cosbench-driver-web/WEB-INF/freemarker/header.ftl index be7c2908..6e604d8b 100644 --- a/dev/cosbench-driver-web/WEB-INF/freemarker/header.ftl +++ b/dev/cosbench-driver-web/WEB-INF/freemarker/header.ftl @@ -1,6 +1,6 @@ <#-- end of header --> diff --git a/dev/cosbench-driver-web/WEB-INF/freemarker/mission.ftl b/dev/cosbench-driver-web/WEB-INF/freemarker/mission.ftl index 5de60b65..17f4c8c5 100644 --- a/dev/cosbench-driver-web/WEB-INF/freemarker/mission.ftl +++ b/dev/cosbench-driver-web/WEB-INF/freemarker/mission.ftl @@ -226,7 +226,7 @@ Error Message Error Code - Occurence Number + Occurrence Number StatckTrace <#list info.errorStatistics.stackTraceAndMessage?keys as trace> @@ -235,10 +235,12 @@ <#else> + <#if trace??> ${info.errorStatistics.stackTraceAndMessage[trace]} ${info.errorStatistics.stackTraceAndErrorCode[trace]} ${info.errorStatistics.stackTraceAndNum[trace]} ${info.errorStatistics.stackTraceAndEntireTrace[trace]?substring(0,50)+"......"} + @@ -273,4 +275,4 @@
<#-- end of main --> <#include "footer.ftl"> - \ No newline at end of file + diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/AbortHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/AbortHandler.java index 10336ee5..7a428185 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/AbortHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/AbortHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.handler; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/AbstractCommandHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/AbstractCommandHandler.java index ada6552c..2f1a3641 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/AbstractCommandHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/AbstractCommandHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.handler; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/CloseHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/CloseHandler.java index c8363a47..e05a216e 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/CloseHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/CloseHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.handler; @@ -21,9 +21,12 @@ import static com.intel.cosbench.model.MissionState.FAILED; import java.io.IOException; +import java.util.ArrayList; import java.util.Arrays; import java.util.HashMap; +import java.util.List; +import com.intel.cosbench.bench.Metrics; import com.intel.cosbench.bench.Report; import com.intel.cosbench.model.MissionInfo; import com.intel.cosbench.model.TaskState; @@ -43,11 +46,18 @@ protected Response process(MissionInfo info) { private Response getResponse(MissionInfo info) { CloseResponse response = new CloseResponse(); Report report = info.getReport(); + List wrReport = new ArrayList(); + for(Report wReport:info.getWorkerReports()){ + for(Metrics metrics : wReport){ + wrReport.add(metrics); + } + } + response.setWrReport(wrReport); response.setReport(Arrays.asList(report.getAllMetrics())); - if (info.getState().equals(FAILED)) - response.setState(TaskState.FAILED); - else - response.setState(TaskState.ACCOMPLISHED); + if (info.getState().equals(FAILED)) + response.setState(TaskState.FAILED); + else + response.setState(TaskState.ACCOMPLISHED); String log = null; try { log = info.getLogManager().getLogAsString(); @@ -56,7 +66,7 @@ private Response getResponse(MissionInfo info) { } response.setErrorStatistics(info.getErrorStatistics().getErrorCodeAndNum()); response.setDriverLog(log); - + return response; } diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/LaunchHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/LaunchHandler.java index 86381f75..d19f70e2 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/LaunchHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/LaunchHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.handler; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/LoginHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/LoginHandler.java index 17e26542..d62cfc20 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/LoginHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/LoginHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.handler; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/MissionHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/MissionHandler.java index 28295754..d828c7a5 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/MissionHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/MissionHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.handler; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/PingHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/PingHandler.java index db9680f3..30434610 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/PingHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/PingHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.handler; @@ -40,35 +40,38 @@ public void setDriver(DriverService driver) { @Override protected Response process(HttpServletRequest req, HttpServletResponse res) - throws Exception { - Scanner scanner = new Scanner(req.getInputStream()); - setSysTime(getControllerTime(scanner)); - - PingResponse response = new PingResponse(); + throws Exception { + Scanner scanner = new Scanner(req.getInputStream()); + Long controllerTime = getControllerTime(scanner); + String setControllerTime = System.getProperty("cosbench.driver.set.controller.time"); + if ("true".equalsIgnoreCase(setControllerTime)) { + setSysTime(controllerTime); + } + PingResponse response = new PingResponse(); DriverInfo info = driver.getDriverInfo(); response.setName(info.getName()); response.setAddress(info.getUrl()); response.setTimeStamp(String.valueOf(System.currentTimeMillis())); return response; } - + private long getControllerTime(Scanner scanner) throws NumberFormatException { - if (!scanner.hasNext()) + if (!scanner.hasNext()) throw new BadRequestException(); - return Long.parseLong(scanner.next()); - } + return Long.parseLong(scanner.next()); + } private void setSysTime(long ctrTime) throws IOException { - DateFormat dateTime = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); - String[] cmd = {"date", "-s", dateTime.format(new Date(ctrTime))}; - String osType = System.getProperty("os.name").toLowerCase(); - if (osType.contains("linux")) { - LOGGER.debug("setting system time {} on driver {}", ctrTime, driver.getDriverInfo().getName()); - Runtime.getRuntime().exec(cmd); - } else { - LOGGER.warn("os type on driver {} is {}!", - driver.getDriverInfo().getName(), osType); - } - } + DateFormat dateTime = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); + String[] cmd = {"date", "-s", dateTime.format(new Date(ctrTime))}; + String osType = System.getProperty("os.name").toLowerCase(); + if (osType.contains("linux")) { + LOGGER.debug("setting system time {} on driver {}", ctrTime, driver.getDriverInfo().getName()); + Runtime.getRuntime().exec(cmd); + } else { + LOGGER.warn("os type on driver {} is {}!", + driver.getDriverInfo().getName(), osType); + } + } } diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/QueryHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/QueryHandler.java index ffd0a534..f9eaaef7 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/QueryHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/QueryHandler.java @@ -1,50 +1,50 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.driver.handler; - -import static com.intel.cosbench.model.MissionState.*; - -import java.util.Arrays; - -import com.intel.cosbench.bench.*; -import com.intel.cosbench.model.MissionInfo; -import com.intel.cosbench.protocol.*; - -public class QueryHandler extends MissionHandler { - - @Override - protected Response process(MissionInfo info) { -// if (isStopped(info.getState())) -// return new Response(false, "mission aleady stopped"); - return getResponse(info); - } - - private Response getResponse(MissionInfo info) { - QueryResponse response = new QueryResponse(); - Snapshot snapshot = info.getSnapshot(); - response.setTime(snapshot.getTimestamp()); - response.setVersion(snapshot.getVersion()); - response.setMinVersion(snapshot.getMinVersion()); - response.setMaxVersion(snapshot.getMaxVersion()); - Report report = snapshot.getReport(); - response.setRunning(info.getState().equals(LAUNCHED)); - response.setReport(Arrays.asList(report.getAllMetrics())); - return response; - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.driver.handler; + +import static com.intel.cosbench.model.MissionState.*; + +import java.util.Arrays; + +import com.intel.cosbench.bench.*; +import com.intel.cosbench.model.MissionInfo; +import com.intel.cosbench.protocol.*; + +public class QueryHandler extends MissionHandler { + + @Override + protected Response process(MissionInfo info) { +// if (isStopped(info.getState())) +// return new Response(false, "mission aleady stopped"); + return getResponse(info); + } + + private Response getResponse(MissionInfo info) { + QueryResponse response = new QueryResponse(); + Snapshot snapshot = info.getSnapshot(); + response.setTime(snapshot.getTimestamp()); + response.setVersion(snapshot.getVersion()); + response.setMinVersion(snapshot.getMinVersion()); + response.setMaxVersion(snapshot.getMaxVersion()); + Report report = snapshot.getReport(); + response.setRunning(info.getState().equals(LAUNCHED)); + response.setReport(Arrays.asList(report.getAllMetrics())); + return response; + } + +} diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/SubmitHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/SubmitHandler.java index 41c2503d..05ba7517 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/SubmitHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/SubmitHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.handler; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/TriggerHandler.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/TriggerHandler.java index fb6dc298..7f485fee 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/TriggerHandler.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/handler/TriggerHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.driver.handler; @@ -35,234 +35,234 @@ import com.intel.cosbench.service.DriverService; public class TriggerHandler extends AbstractCommandHandler { - - private int currPID = 0; - private int xferPID = 0; - private boolean isEnable = false; - protected static String scriptsDir = "scripts/"; - protected DriverService driver; - protected String scriptLog; - - + + private int currPID = 0; + private int xferPID = 0; + private boolean isEnable = false; + protected static String scriptsDir = "scripts/"; + protected DriverService driver; + protected String scriptLog; + + public void setDriver(DriverService driver) { this.driver = driver; } - + @Override protected Response process(HttpServletRequest req, HttpServletResponse res) throws Exception { - scriptLog = ""; - Scanner scanner = new Scanner(req.getInputStream()); - String trigger = getTrigger(scanner); - runTrigger(trigger); - + scriptLog = ""; + Scanner scanner = new Scanner(req.getInputStream()); + String trigger = getTrigger(scanner); + runTrigger(trigger); + return createResponse(); } - + private String getTrigger(Scanner scanner) { - if (!scanner.hasNext()) + if (!scanner.hasNext()) LOGGER.error("bad request exception"); - String trigger = scanner.next(); - if (trigger == null) - LOGGER.error("no found exception"); + String trigger = scanner.next(); + if (trigger == null) + LOGGER.error("no found exception"); return trigger; - } - + } + private void runTrigger(String trigger) { - String[] cmdArr = getCmdArray(trigger); - if (cmdArr == null) { - return; - } - LOGGER.info("executing trigger command line : {} on {}", cmdArr, getDriverName()); - String osType = System.getProperty("os.name").toLowerCase(); - if (osType.contains("linux")) { - try { - Process process = Runtime.getRuntime().exec(cmdArr); - getPID(process); - InputStream is = process.getInputStream(); - BufferedReader buff = new BufferedReader(new InputStreamReader(is)); - String line = null; - while ((line = buff.readLine()) != null) - scriptLog += line + "\n"; - process.waitFor(); - } catch (Exception e) { - LOGGER.error("execute trigger command failed!"); - return; - } - } else { - LOGGER.warn("can not execute trigger on {}, the OS type({}) isn't linux!", getDriverName(), osType); - return; - } - if (!isEnable) { - if (!isTriggerKilled(currPID)) { - killByPID(currPID); - } - if (!isTriggerKilled(xferPID)) - killByPID(xferPID); - } - } - + String[] cmdArr = getCmdArray(trigger); + if (cmdArr == null) { + return; + } + LOGGER.info("executing trigger command line : {} on {}", cmdArr, getDriverName()); + String osType = System.getProperty("os.name").toLowerCase(); + if (osType.contains("linux")) { + try { + Process process = Runtime.getRuntime().exec(cmdArr); + getPID(process); + InputStream is = process.getInputStream(); + BufferedReader buff = new BufferedReader(new InputStreamReader(is)); + String line = null; + while ((line = buff.readLine()) != null) + scriptLog += line + "\n"; + process.waitFor(); + } catch (Exception e) { + LOGGER.error("execute trigger command failed!"); + return; + } + } else { + LOGGER.warn("can not execute trigger on {}, the OS type({}) isn't linux!", getDriverName(), osType); + return; + } + if (!isEnable) { + if (!isTriggerKilled(currPID)) { + killByPID(currPID); + } + if (!isTriggerKilled(xferPID)) + killByPID(xferPID); + } + } + private String[] getCmdArray(String trigger) { - trigger.replace(" ", ""); - String[] triggerArr = StringUtils.split(trigger, ','); - if (triggerArr == null) { - LOGGER.warn("trigger command array is empty!"); - return null; - } - if (triggerArr[0].equals("killTrigger")) { - this.isEnable = false; - if (triggerArr.length != 3) { - LOGGER.error("kill-trigger command line is illegal!"); - return null; - } - xferPID = parsePID(triggerArr[1]); - String fileName = triggerArr[2]; - String filePath = scriptsDir + fileName; - File tempPath = new File(filePath); - if (!tempPath.exists() || !tempPath.isFile()) { - LOGGER.warn("trigger file {} dosen't exist on {}!", filePath, getDriverName()); - return null; - } - return new String[]{"/bin/sh", filePath, "-k"}; - } else if (triggerArr[0].equals("enableTrigger")) { - this.isEnable = true; - String fileName = triggerArr[1]; - String filePath = scriptsDir + fileName; - File tempPath = new File(filePath); - if (!tempPath.exists() || !tempPath.isFile()) { - LOGGER.warn("trigger file {} dosen't exist on {}!", filePath, getDriverName()); - return null; - } - String[] cmdArr = new String[triggerArr.length]; - cmdArr[0] = "/bin/sh"; - cmdArr[1] = filePath; - for (int i = 2; i < triggerArr.length; i++) { - cmdArr[i] = triggerArr[i]; - } - return cmdArr; - } else { - LOGGER.error("trigger command line is illegal!"); - return null; - } - } - + trigger.replace(" ", ""); + String[] triggerArr = StringUtils.split(trigger, ','); + if (triggerArr == null) { + LOGGER.warn("trigger command array is empty!"); + return null; + } + if (triggerArr[0].equals("killTrigger")) { + this.isEnable = false; + if (triggerArr.length != 3) { + LOGGER.error("kill-trigger command line is illegal!"); + return null; + } + xferPID = parsePID(triggerArr[1]); + String fileName = triggerArr[2]; + String filePath = scriptsDir + fileName; + File tempPath = new File(filePath); + if (!tempPath.exists() || !tempPath.isFile()) { + LOGGER.warn("trigger file {} dosen't exist on {}!", filePath, getDriverName()); + return null; + } + return new String[]{"/bin/sh", filePath, "-k"}; + } else if (triggerArr[0].equals("enableTrigger")) { + this.isEnable = true; + String fileName = triggerArr[1]; + String filePath = scriptsDir + fileName; + File tempPath = new File(filePath); + if (!tempPath.exists() || !tempPath.isFile()) { + LOGGER.warn("trigger file {} dosen't exist on {}!", filePath, getDriverName()); + return null; + } + String[] cmdArr = new String[triggerArr.length]; + cmdArr[0] = "/bin/sh"; + cmdArr[1] = filePath; + for (int i = 2; i < triggerArr.length; i++) { + cmdArr[i] = triggerArr[i]; + } + return cmdArr; + } else { + LOGGER.error("trigger command line is illegal!"); + return null; + } + } + private int parsePID(String str) { - int num = 0; - try { - num = Integer.parseInt(str); - } catch (NumberFormatException e) { - LOGGER.error("can not parse PID from String to Int {}!", str); - } - return num; - } - + int num = 0; + try { + num = Integer.parseInt(str); + } catch (NumberFormatException e) { + LOGGER.error("can not parse PID from String to Int {}!", str); + } + return num; + } + private void getPID(Process process) { - Field field = null; - if (!process.getClass().getName().equals("java.lang.UNIXProcess")) { - LOGGER.error("failed to get PID by {}", process.getClass().getName()); - return; - } - try { - field = process.getClass().getDeclaredField("pid"); - field.setAccessible(true); - currPID = (Integer) field.get(process); - } catch (Exception e) { - LOGGER.error("get PID failed!"); - } - if (isEnable) - xferPID = currPID; - LOGGER.debug("current PID is: {}", currPID); - } - + Field field = null; + if (!process.getClass().getName().equals("java.lang.UNIXProcess")) { + LOGGER.error("failed to get PID by {}", process.getClass().getName()); + return; + } + try { + field = process.getClass().getDeclaredField("pid"); + field.setAccessible(true); + currPID = (Integer) field.get(process); + } catch (Exception e) { + LOGGER.error("get PID failed!"); + } + if (isEnable) + xferPID = currPID; + LOGGER.debug("current PID is: {}", currPID); + } + private boolean isTriggerKilled(int pid) { - List pids = getAllPID(pid); - if (pids != null && pids.size() >= 1) - return false; - return true; - } + List pids = getAllPID(pid); + if (pids != null && pids.size() >= 1) + return false; + return true; + } private void killByPID(int pid) { - List pidList = getAllPID(pid); - if (pidList == null || pidList.isEmpty()) { - //LOGGER.debug("pid {} have been killed", pid); - return; - } - if (pidList.size() == 1) { - //LOGGER.debug("pid {} have no child process", pid); - } else { - //LOGGER.debug("pid {} have child process {}", pid, pidList); - for (int i = 1; i < pidList.size(); i++) { - killByPID(pidList.get(i)); - } - } - runKill9(pid); - } + List pidList = getAllPID(pid); + if (pidList == null || pidList.isEmpty()) { + //LOGGER.debug("pid {} have been killed", pid); + return; + } + if (pidList.size() == 1) { + //LOGGER.debug("pid {} have no child process", pid); + } else { + //LOGGER.debug("pid {} have child process {}", pid, pidList); + for (int i = 1; i < pidList.size(); i++) { + killByPID(pidList.get(i)); + } + } + runKill9(pid); + } private List getAllPID(int pid) { - if (pid <= 0) { - LOGGER.warn("pid is illegal: {}", pid); - return null; - } - List pidList = new ArrayList(); - String cmdLine = "ps -ef | grep " + pid + " | awk '{print $2}'"; - try { - Process process = Runtime.getRuntime().exec(new String[]{"/bin/sh", "-c", cmdLine}); - InputStream is = process.getInputStream(); - BufferedReader buff = new BufferedReader(new InputStreamReader(is)); + if (pid <= 0) { + LOGGER.warn("pid is illegal: {}", pid); + return null; + } + List pidList = new ArrayList(); + String cmdLine = "ps -ef | grep " + pid + " | awk '{print $2}'"; + try { + Process process = Runtime.getRuntime().exec(new String[]{"/bin/sh", "-c", cmdLine}); + InputStream is = process.getInputStream(); + BufferedReader buff = new BufferedReader(new InputStreamReader(is)); - String line = null; - while ((line = buff.readLine()) != null) { - int tmp = 0; - if ((tmp = parsePID(line.trim())) <= 0){ - LOGGER.error("buffer line is illegal: {}", line); - continue; - } - //LOGGER.debug("child pid of {} : {}", pid, tmp); - pidList.add(new Integer(tmp)); - } - process.waitFor(); - } catch (Exception e) { - LOGGER.error("get all PID failed!"); - return null; - } - if (pidList == null || pidList.isEmpty()) - return null; - pidList.remove(pidList.size()-1); - pidList.remove(pidList.size()-1); - return pidList; - } + String line = null; + while ((line = buff.readLine()) != null) { + int tmp = 0; + if ((tmp = parsePID(line.trim())) <= 0){ + LOGGER.error("buffer line is illegal: {}", line); + continue; + } + //LOGGER.debug("child pid of {} : {}", pid, tmp); + pidList.add(new Integer(tmp)); + } + process.waitFor(); + } catch (Exception e) { + LOGGER.error("get all PID failed!"); + return null; + } + if (pidList == null || pidList.isEmpty()) + return null; + pidList.remove(pidList.size()-1); + pidList.remove(pidList.size()-1); + return pidList; + } private void runKill9(int pid) { - String osType = System.getProperty("os.name").toLowerCase(); - if (osType.contains("linux")) { - try { - Runtime.getRuntime().exec(new String[]{"kill","-9",Integer.toString(pid)}); - LOGGER.debug("pid {} have been killed directly", pid); - } catch (Exception e) { - LOGGER.error("run failed!", pid); - } - } else { - LOGGER.warn("can not run kill command on {}, the OS type({}) isn't linux!", getDriverName(), osType); - } - } - + String osType = System.getProperty("os.name").toLowerCase(); + if (osType.contains("linux")) { + try { + Runtime.getRuntime().exec(new String[]{"kill","-9",Integer.toString(pid)}); + LOGGER.debug("pid {} have been killed directly", pid); + } catch (Exception e) { + LOGGER.error("run failed!", pid); + } + } else { + LOGGER.warn("can not run kill command on {}, the OS type({}) isn't linux!", getDriverName(), osType); + } + } + private String getDriverName() { - String name = driver.getDriverInfo().getName(); - if ( name != null && !name.isEmpty()) - return name; - try { - name = InetAddress.getLocalHost().getHostName(); - } catch (Exception e) { - LOGGER.warn("can not get local hostname on {}", driver.getDriverInfo().getUrl()); - } - return name; - } - + String name = driver.getDriverInfo().getName(); + if ( name != null && !name.isEmpty()) + return name; + try { + name = InetAddress.getLocalHost().getHostName(); + } catch (Exception e) { + LOGGER.warn("can not get local hostname on {}", driver.getDriverInfo().getUrl()); + } + return name; + } + private Response createResponse() { TriggerResponse response = new TriggerResponse(); response.setPID(isEnable ? Integer.toString(xferPID) : "0"); response.setScriptLog(scriptLog); return response; - } + } } diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/AbortMissionController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/AbortMissionController.java index 9bb7996f..1cdffcf3 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/AbortMissionController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/AbortMissionController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/CloseMissionController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/CloseMissionController.java index ba23a34d..6aa7edd4 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/CloseMissionController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/CloseMissionController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/DownloadLogController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/DownloadLogController.java index cc4634a8..70671fa0 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/DownloadLogController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/DownloadLogController.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.driver.web; import java.io.IOException; @@ -12,7 +30,7 @@ import com.intel.cosbench.model.MissionInfo; public class DownloadLogController extends MissionPageController{ - private static final View LOG = new LogView(); + private static final View LOG = new LogView(); private static class LogView implements View { @@ -35,11 +53,11 @@ public void render(Map model, HttpServletRequest req, protected ModelAndView process(MissionInfo info) { String log = ""; - try { - log = info.getLogManager().getLogAsString(); - } catch (IOException e) { - e.printStackTrace(); - } + try { + log = info.getLogManager().getLogAsString(); + } catch (IOException e) { + e.printStackTrace(); + } return new ModelAndView(LOG, "log", log); } diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/IndexPageController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/IndexPageController.java index 4ac808dc..ef2f01fe 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/IndexPageController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/IndexPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; @@ -26,7 +26,7 @@ /** * The home page of driver web console. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/LaunchMissionController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/LaunchMissionController.java index 305f72ae..8236dc6c 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/LaunchMissionController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/LaunchMissionController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/MissionPageController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/MissionPageController.java index bc34a97e..130c8184 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/MissionPageController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/MissionPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; @@ -55,6 +55,7 @@ protected ModelAndView process(String id) { protected ModelAndView process(MissionInfo info) { ModelAndView result = new ModelAndView("mission"); result.addObject("info", info); + result.addObject("dInfo", driver.getDriverInfo()); result.addObject("isStopped", isStopped(info.getState())); result.addObject("isRunning", isRunning(info.getState())); result.addObject("toBeAuthed", allowAuth(info.getState())); diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/MissionSubmissionController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/MissionSubmissionController.java index fbf6b38b..bb015d44 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/MissionSubmissionController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/MissionSubmissionController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; @@ -65,6 +65,7 @@ private InputStream retrieveConfigStream(HttpServletRequest request) private ModelAndView createErrResult(String msg) { ModelAndView result = new ModelAndView("submit"); + result.addObject("dInfo", driver.getDriverInfo()); result.addObject("aInfos", driver.getActiveMissions()); result.addObject("error", msg); return result; @@ -72,6 +73,7 @@ private ModelAndView createErrResult(String msg) { private ModelAndView createSuccResult(String id) { ModelAndView result = new ModelAndView("submit"); + result.addObject("dInfo", driver.getDriverInfo()); result.addObject("aInfos", driver.getActiveMissions()); result.addObject("submitted", "your mission has been accepted"); result.addObject("id", id); diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/PerformLoginController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/PerformLoginController.java index daacc28e..e965a6a5 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/PerformLoginController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/PerformLoginController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/SubmitPageController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/SubmitPageController.java index eefa39a1..9e8cd827 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/SubmitPageController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/SubmitPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; @@ -37,7 +37,9 @@ public void setDriver(DriverService driver) { protected ModelAndView process(HttpServletRequest req, HttpServletResponse res) throws Exception { MissionInfo[] aInfos = driver.getActiveMissions(); - return new ModelAndView("submit", "aInfos", aInfos); + ModelAndView result = new ModelAndView("submit", "aInfos", aInfos); + result.addObject("dInfo", driver.getDriverInfo()); + return result; } } diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/TriggerController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/TriggerController.java index 23db135c..c61f7da9 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/TriggerController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/TriggerController.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.driver.web; import javax.servlet.http.HttpServletRequest; @@ -9,7 +27,7 @@ import com.intel.cosbench.web.AbstractController; public class TriggerController extends AbstractController { - protected DriverService driver; + protected DriverService driver; public void setDriver(DriverService driver) { this.driver = driver; diff --git a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/WorkersPageController.java b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/WorkersPageController.java index 1492cfce..aaf96067 100644 --- a/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/WorkersPageController.java +++ b/dev/cosbench-driver-web/src/com/intel/cosbench/driver/web/WorkersPageController.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.web; @@ -30,8 +30,7 @@ public class WorkersPageController extends AbstractController { - protected DriverService driver; - + protected DriverService driver; public void setDriver(DriverService driver) { this.driver = driver; } @@ -45,7 +44,8 @@ protected ModelAndView process(HttpServletRequest req, MissionInfo info = driver.getMissionInfo(id); if (info == null) throw new NotFoundException(); - return createResult(info); + ModelAndView result = createResult(info).addObject("dInfo", driver.getDriverInfo()); + return result; } private static ModelAndView createResult(MissionInfo info) { diff --git a/dev/cosbench-driver/META-INF/MANIFEST.MF b/dev/cosbench-driver/META-INF/MANIFEST.MF index 9cd80104..b0ea7f67 100644 --- a/dev/cosbench-driver/META-INF/MANIFEST.MF +++ b/dev/cosbench-driver/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Driver Bundle Bundle-SymbolicName: cosbench-driver -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AbstractAgent.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AbstractAgent.java index c928d453..f5039de1 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AbstractAgent.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AbstractAgent.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.agent; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/Agent.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/Agent.java index b3bbde10..439dea7f 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/Agent.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/Agent.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.agent; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AgentException.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AgentException.java index 4aff02c7..c9e84af6 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AgentException.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AgentException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.agent; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/Agents.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/Agents.java index 2685ca6e..441d64d3 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/Agents.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/Agents.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.agent; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AuthAgent.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AuthAgent.java index 3803b8a9..0afe9615 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AuthAgent.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/AuthAgent.java @@ -1,142 +1,143 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.driver.agent; - -import java.util.Random; - -import org.apache.commons.lang.math.RandomUtils; - -import com.intel.cosbench.api.auth.*; -import com.intel.cosbench.api.context.AuthContext; -import com.intel.cosbench.api.auth.AuthConstants; -import com.intel.cosbench.api.storage.StorageAPI; -import com.intel.cosbench.driver.util.AuthCachePool; -import com.intel.cosbench.log.Logger; -import com.intel.cosbench.service.AbortedException; - -class AuthAgent extends AbstractAgent { - - private int loginAttempts; // number of retries - - public AuthAgent() { - /* empty */ - } - - public void setLoginAttempts(int loginAttempts) { - this.loginAttempts = loginAttempts; - } - - @Override - protected void execute() { - Logger logger = getMissionLogger(); - try { - /* - * Here we prepare the storage with the authentication object (e.g. - * a user token) retrieved from an login operation. Some storage - * implementations such as Swift and S3 do require such information, - * others don't. But we will do this anyway. - */ - StorageAPI storageApi = workerContext.getStorageApi(); - storageApi.setAuthContext(login()); - - } catch (AuthInterruptedException ie) { - throw new AbortedException(); - } catch (AuthBadException be) { - logger.error("bad username and password", be); - throw new AgentException(); // mark error - } catch (AuthException e) { - if (loginAttempts == 1) - logger.error("unable to login", e); - else - logger.error("still unable to login", e); - logger.error("fail to login with {} attempt(s)", loginAttempts); - throw new AgentException(); // mark error - } finally { - AuthAPI authApi = workerContext.getAuthApi(); - authApi.dispose(); // early dispose connection - } - int idx = workerContext.getIndex(); - logger.debug("worker {} has been successfully authed", idx); - } - - private AuthContext login() { - Logger logger = getMissionLogger(); - AuthAPI authApi = workerContext.getAuthApi(); - int attempts = 0; - while (attempts++ < loginAttempts - 1) - try { - return tryLogin(); - } catch (AuthInterruptedException ie) { - throw ie; // do not mask this one - } catch (AuthBadException be) { - throw be; // do not mask this one, either - } catch (AuthException e) { - logger.error("unable to login, will try again later", e); - sleepForSometime(); - } - return tryLogin(); // the very last attempt! - } - - private AuthContext tryLogin() { - Logger logger = getMissionLogger(); - logger.debug("begin to login, will attempt {} times", loginAttempts); - - AuthAPI authApi = workerContext.getAuthApi(); - AuthContext import_context = authApi.getParms(); - AuthContext auth_context; - String id = import_context.getID(); - - boolean caching = import_context.getBoolean(AuthConstants.CACHING_KEY, AuthConstants.CACHING_DEFAULT); - logger.debug("input auth context is {} with caching={}", import_context.toString(), caching); - if(caching) { // auth caching is enabled - // check if auth context is already cached. - logger.debug("auth caching is enabled, will query cache pool with id={}", id); - synchronized(AuthCachePool.getInstance()) { - if(AuthCachePool.getInstance().containsKey(id)) {// already cached - auth_context = AuthCachePool.getInstance().get(id); - logger.debug("auth context for id={} is found as {}", id, auth_context); - } - else { // not found - logger.debug("auth context for id={} is not found, will try to login", id); - auth_context = authApi.login(); - if(auth_context != null) // the authentication mechanism is embedded into storage adapter - { - logger.debug("login is successful, auth context for id={} will be cacahed as {}", id, auth_context); - AuthCachePool.getInstance().put(id, auth_context); - }else { // the authentication mechanism is embedded into storage adapter - logger.info("no auth context required."); - } - } - } - } - else - auth_context = authApi.login(); - - return auth_context; - } - - private void sleepForSometime() { - Random random = workerContext.getRandom(); - int time = (10 + RandomUtils.nextInt(random, 40)) * 100; - try { - Thread.sleep(time); - } catch (InterruptedException e) { - throw new AuthInterruptedException(e); - } - Logger logger = getMissionLogger(); - logger.debug("has waited for {} ms, will make a new attempt", time); - } - +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.driver.agent; + +import java.util.Random; + +import org.apache.commons.lang.math.RandomUtils; + +import com.intel.cosbench.api.auth.*; +import com.intel.cosbench.api.context.AuthContext; +import com.intel.cosbench.api.auth.AuthConstants; +import com.intel.cosbench.api.storage.StorageAPI; +import com.intel.cosbench.driver.util.AuthCachePool; +import com.intel.cosbench.log.Logger; +import com.intel.cosbench.service.AbortedException; + +class AuthAgent extends AbstractAgent { + + private int loginAttempts; // number of retries + + public AuthAgent() { + /* empty */ + } + + public void setLoginAttempts(int loginAttempts) { + this.loginAttempts = loginAttempts; + } + + @Override + protected void execute() { + Logger logger = getMissionLogger(); + try { + /* + * Here we prepare the storage with the authentication object (e.g. + * a user token) retrieved from an login operation. Some storage + * implementations such as Swift and S3 do require such information, + * others don't. But we will do this anyway. + */ + StorageAPI storageApi = workerContext.getStorageApi(); + storageApi.setAuthContext(login()); + + } catch (AuthInterruptedException ie) { + throw new AbortedException(); + } catch (AuthBadException be) { + logger.error("bad username and password", be); + throw new AgentException(); // mark error + } catch (AuthException e) { + if (loginAttempts == 1) + logger.error("unable to login", e); + else + logger.error("still unable to login", e); + logger.error("fail to login with {} attempt(s)", loginAttempts); + throw new AgentException(); // mark error + } finally { + AuthAPI authApi = workerContext.getAuthApi(); + authApi.dispose(); // early dispose connection + } + int idx = workerContext.getIndex(); + logger.debug("worker {} has been successfully authed", idx); + } + + private AuthContext login() { + Logger logger = getMissionLogger(); + AuthAPI authApi = workerContext.getAuthApi(); + + int attempts = 0; + while (attempts++ < loginAttempts - 1) + try { + return tryLogin(); + } catch (AuthInterruptedException ie) { + throw ie; // do not mask this one + } catch (AuthBadException be) { + throw be; // do not mask this one, either + } catch (AuthException e) { + logger.error("unable to login, will try again later", e); + sleepForSometime(); + } + return tryLogin(); // the very last attempt! + } + + private AuthContext tryLogin() { + Logger logger = getMissionLogger(); + logger.debug("begin to login, will attempt {} times", loginAttempts); + + AuthAPI authApi = workerContext.getAuthApi(); + AuthContext import_context = authApi.getParms(); + AuthContext auth_context; + String id = import_context.getID(); + + boolean caching = import_context.getBoolean(AuthConstants.CACHING_KEY, AuthConstants.CACHING_DEFAULT); + logger.debug("input auth context is {} with caching={}", import_context.toString(), caching); + if(caching) { // auth caching is enabled + // check if auth context is already cached. + logger.debug("auth caching is enabled, will query cache pool with id={}", id); + synchronized(AuthCachePool.getInstance()) { + if(AuthCachePool.getInstance().containsKey(id)) {// already cached + auth_context = AuthCachePool.getInstance().get(id); + logger.debug("auth context for id={} is found as {}", id, auth_context); + } + else { // not found + logger.debug("auth context for id={} is not found, will try to login", id); + auth_context = authApi.login(); + if(auth_context != null) // the authentication mechanism is embedded into storage adapter + { + logger.debug("login is successful, auth context for id={} will be cacahed as {}", id, auth_context); + AuthCachePool.getInstance().put(id, auth_context); + }else { // the authentication mechanism is embedded into storage adapter + logger.info("no auth context required."); + } + } + } + } + else + auth_context = authApi.login(); + + return auth_context; + } + + private void sleepForSometime() { + Random random = workerContext.getRandom(); + int time = (10 + RandomUtils.nextInt(random, 40)) * 100; + try { + Thread.sleep(time); + } catch (InterruptedException e) { + throw new AuthInterruptedException(e); + } + Logger logger = getMissionLogger(); + logger.debug("has waited for {} ms, will make a new attempt", time); + } + } \ No newline at end of file diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/WatchDog.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/WatchDog.java index cb3f0763..9e73af69 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/WatchDog.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/WatchDog.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.agent; @@ -65,8 +65,8 @@ public void run() { } if (!cancel) { workerContext.getStorageApi().abort(); - workerContext.disposeRuntime(); - + workerContext.disposeRuntime(); + LOGGER.debug("work agent {} has been alerted for timeout", idx); } else LOGGER.debug("work agent {} has completed before timeout", idx); diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/WorkAgent.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/WorkAgent.java index eebd02a2..89a98e4c 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/WorkAgent.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/agent/WorkAgent.java @@ -1,309 +1,309 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.driver.agent; - -import static com.intel.cosbench.bench.Mark.*; - -import com.intel.cosbench.bench.ErrorStatistics; - -import java.util.*; - -import javax.naming.AuthenticationException; - -import com.intel.cosbench.api.auth.AuthBadException; -import com.intel.cosbench.api.auth.AuthException; -import com.intel.cosbench.api.context.AuthContext; -import com.intel.cosbench.api.storage.StorageAPI; -import com.intel.cosbench.bench.*; -import com.intel.cosbench.config.Mission; -import com.intel.cosbench.driver.model.*; -import com.intel.cosbench.driver.operator.*; -import com.intel.cosbench.driver.util.AuthCachePool; -import com.intel.cosbench.driver.util.OperationPicker; -import com.intel.cosbench.log.Logger; -import com.intel.cosbench.service.AbortedException; - -class WorkAgent extends AbstractAgent implements Session, OperationListener { - - private long start; /* agent startup time */ - private long begin; /* effective workload startup time */ - private long end; /* effective workload shut-down time */ - private long timeout; /* expected agent stop time */ - - private long lop; /* last operation performed */ - private long lbegin; /* last sample emitted */ - private long lsample; /* last sample collected */ - private long lrsample; /* last sample collected during runtime */ - private long frsample; /* first sample emitted during runtime */ - - private long curr; /* current time */ - private long lcheck; /* last check point time */ - private long check; /* next check point time */ - private long interval; /* interval between check points */ - - private int totalOps; /* total operations to be performed */ -// private int op_count; - private long totalBytes; /* total bytes to be transferred */ - private boolean has_histo; /* collect response time histogram data or not */ - - private OperationPicker operationPicker; - private OperatorRegistry operatorRegistry; - -// private boolean isFinished = false; - private WatchDog dog = new WatchDog(); - - private Status currMarks = new Status(); /* for snapshots */ - private Status globalMarks = new Status(); /* for the final report */ - - public WorkAgent() { - /* empty */ - } - - @Override - public void setWorkerContext(WorkerContext workerContext) { - super.setWorkerContext(workerContext); - this.has_histo = workerContext.getMission().hasHisto(); - - dog.setWorkerContext(workerContext); - } - - public void setOperationPicker(OperationPicker operationPicker) { - this.operationPicker = operationPicker; - } - - public void setOperatorRegistry(OperatorRegistry operatorRegistry) { - this.operatorRegistry = operatorRegistry; - } - - @Override - public int getIndex() { - return workerContext.getIndex(); - } - - @Override - public int getTotalWorkers() { - return workerContext.getMission().getTotalWorkers(); - } - - @Override - public Random getRandom() { - return workerContext.getRandom(); - } - - @Override - public StorageAPI getApi() { - return workerContext.getStorageApi(); - } - - @Override - public Logger getLogger() { - return workerContext.getLogger(); - } - - public ErrorStatistics getErrorStatistics(){ - return workerContext.getErrorStatistics(); - } - - @Override - public OperationListener getListener() { - return this; - } - - @Override - protected void execute() { - initTimes(); - initLimites(); - initMarks(); - dog.watch(timeout); - try { - doWork(); // launch work - } finally { - dog.dismiss(); - } - /* work agent has completed execution successfully */ - } - - private void initTimes() { - Mission mission = workerContext.getMission(); - interval = mission.getInterval(); - lcheck = curr = start = System.currentTimeMillis(); - check = lcheck + interval * 1000; - begin = start; - timeout = 0L; - lop = lrsample = lsample = start; - frsample = lbegin = end = Long.MAX_VALUE; - } - - private void initLimites() { - Mission mission = workerContext.getMission(); - totalOps = mission.getTotalOps() / mission.getTotalWorkers(); - totalBytes = mission.getTotalBytes() / mission.getTotalWorkers(); - if (mission.getRuntime() == 0) - return; - begin = start + mission.getRampup() * 1000; - end = begin + mission.getRuntime() * 1000; - timeout = end + mission.getRampdown() * 1000; - } - - private void initMarks() { - Set types = new LinkedHashSet(); - for (OperatorContext op : operatorRegistry) - types.add(getMarkType(op.getId(), op.getOpType(), op.getSampleType(), op.getName())); - for (String type : types) - currMarks.addMark(newMark(type)); - for (String type : types) - globalMarks.addMark(newMark(type)); - } - - private void doWork() { - doSnapshot(); - while (!workerContext.isFinished()) - try { - performOperation(); - }catch (AbortedException ae) { - if (lrsample > frsample) - doSummary(); - workerContext.setFinished(true); - } - doSnapshot(); - } - - - private void performOperation() { - if(workerContext.getAuthApi() == null || workerContext.getStorageApi() == null) - throw new AbortedException(); - if(! workerContext.getStorageApi().isAuthValid()) - reLogin(); - lbegin = System.currentTimeMillis(); - Random random = workerContext.getRandom(); - String op = operationPicker.pickOperation(random); - OperatorContext context = operatorRegistry.getOperator(op); - try{ - context.getOperator().operate(this); - }catch(AuthException ae) { - reLogin(); - } - } - - @Override - public void onSampleCreated(Sample sample) { - curr = sample.getTimestamp().getTime(); - String type = getMarkType(sample.getOpId(), sample.getOpType(), - sample.getSampleType(), sample.getOpName()); - currMarks.getMark(type).addSample(sample); - if (lbegin >= begin && lbegin < end && curr > begin && curr <= end) { - globalMarks.getMark(type).addSample(sample); - operatorRegistry.getOperator(sample.getOpId()).addSample(sample); - if (lbegin < frsample) - frsample = lbegin; // first sample emitted during runtime - lrsample = curr; // last sample collected during runtime - } - lsample = curr; // last sample collected - trySnapshot(); // make a snapshot if necessary - } - - private void trySnapshot() { - if (lsample < check) - return; - doSnapshot(); - lcheck = System.currentTimeMillis(); - check = lcheck + interval * 1000; - } - - private void doSnapshot() { - long window = lsample - lcheck; - Report report = new Report(); - for (Mark mark : currMarks) { - report.addMetrics(Metrics.convert(mark, window)); - mark.clear(); - } - Snapshot snapshot = new Snapshot(report); - workerContext.setSnapshot(snapshot); - } - - @Override - public void onOperationCompleted(Result result) { - curr = result.getTimestamp().getTime(); -/* */ - String type = getMarkType(result.getOpId(), result.getOpType(), - result.getSampleType(), result.getOpName()); - currMarks.getMark(type).addOperation(result); - if (lop >= begin && lop < end && curr > begin && curr <= end) - globalMarks.getMark(type).addOperation(result); -/* */ - lop = curr; // last operation performed - trySummary(); // make a summary report if necessary - } - - private void trySummary() { - if ((timeout <= 0 || curr < timeout) // timeout - && (totalOps <= 0 || getTotalOps() < totalOps) // operations - && (totalBytes <= 0 || getTotalBytes() < totalBytes)) // bytes - return; // not finished - doSummary(); - - workerContext.setFinished(true); - } - - private void doSummary() { -/* */ - long window = lrsample - frsample; - Report report = new Report(); - for (Mark mark : globalMarks) - report.addMetrics(Metrics.convert(mark, window)); - workerContext.setReport(report); -/* */ - } - - private int getTotalOps() { -// return ++op_count; - - int sum = 0; - for (Mark mark : globalMarks) - sum += mark.getTotalOpCount(); - return sum; - } - - private long getTotalBytes() { - long bytes = 0; - for (Mark mark : globalMarks) - bytes += mark.getByteCount(); - return bytes; - } - public void reLogin() { - LOGGER.debug("WorkAgent {} auth failed, now relogin",workerContext.getIndex()); - AuthContext authContext = workerContext.getStorageApi().getAuthContext(); - synchronized (AuthCachePool.getInstance()) { - AuthCachePool.getInstance().remove(authContext.getID()); - } - try{ - workerContext.getAuthApi().init(); - authContext = workerContext.getAuthApi().login(); - workerContext.getStorageApi().setAuthContext(authContext); - synchronized (AuthCachePool.getInstance()) { - AuthCachePool.getInstance().put(authContext.getID(), authContext); - } - LOGGER.debug("WorkAgent {} relogin successfully",workerContext.getIndex()); - }catch(AuthException ae) { - workerContext.getAuthApi().dispose(); - LOGGER.error("agent "+workerContext.getIndex()+" failed to login",ae); - throw new AgentException(); - } - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.driver.agent; + +import static com.intel.cosbench.bench.Mark.*; + +import com.intel.cosbench.bench.ErrorStatistics; + +import java.util.*; + +import javax.naming.AuthenticationException; + +import com.intel.cosbench.api.auth.AuthBadException; +import com.intel.cosbench.api.auth.AuthException; +import com.intel.cosbench.api.context.AuthContext; +import com.intel.cosbench.api.storage.StorageAPI; +import com.intel.cosbench.bench.*; +import com.intel.cosbench.config.Mission; +import com.intel.cosbench.driver.model.*; +import com.intel.cosbench.driver.operator.*; +import com.intel.cosbench.driver.util.AuthCachePool; +import com.intel.cosbench.driver.util.OperationPicker; +import com.intel.cosbench.log.Logger; +import com.intel.cosbench.service.AbortedException; + +class WorkAgent extends AbstractAgent implements Session, OperationListener { + + private long start; /* agent startup time */ + private long begin; /* effective workload startup time */ + private long end; /* effective workload shut-down time */ + private long timeout; /* expected agent stop time */ + + private long lop; /* last operation performed */ + private long lbegin; /* last sample emitted */ + private long lsample; /* last sample collected */ + private long lrsample; /* last sample collected during runtime */ + private long frsample; /* first sample emitted during runtime */ + + private long curr; /* current time */ + private long lcheck; /* last check point time */ + private long check; /* next check point time */ + private long interval; /* interval between check points */ + + private int totalOps; /* total operations to be performed */ +// private int op_count; + private long totalBytes; /* total bytes to be transferred */ + private boolean has_histo; /* collect response time histogram data or not */ + + private OperationPicker operationPicker; + private OperatorRegistry operatorRegistry; + +// private boolean isFinished = false; + private WatchDog dog = new WatchDog(); + + private Status currMarks = new Status(); /* for snapshots */ + private Status globalMarks = new Status(); /* for the final report */ + + public WorkAgent() { + /* empty */ + } + + @Override + public void setWorkerContext(WorkerContext workerContext) { + super.setWorkerContext(workerContext); + this.has_histo = workerContext.getMission().hasHisto(); + + dog.setWorkerContext(workerContext); + } + + public void setOperationPicker(OperationPicker operationPicker) { + this.operationPicker = operationPicker; + } + + public void setOperatorRegistry(OperatorRegistry operatorRegistry) { + this.operatorRegistry = operatorRegistry; + } + + @Override + public int getIndex() { + return workerContext.getIndex(); + } + + @Override + public int getTotalWorkers() { + return workerContext.getMission().getTotalWorkers(); + } + + @Override + public Random getRandom() { + return workerContext.getRandom(); + } + + @Override + public StorageAPI getApi() { + return workerContext.getStorageApi(); + } + + @Override + public Logger getLogger() { + return workerContext.getLogger(); + } + + public ErrorStatistics getErrorStatistics(){ + return workerContext.getErrorStatistics(); + } + + @Override + public OperationListener getListener() { + return this; + } + + @Override + protected void execute() { + initTimes(); + initLimites(); + initMarks(); + dog.watch(timeout); + try { + doWork(); // launch work + } finally { + dog.dismiss(); + } + /* work agent has completed execution successfully */ + } + + private void initTimes() { + Mission mission = workerContext.getMission(); + interval = mission.getInterval(); + lcheck = curr = start = System.currentTimeMillis(); + check = lcheck + interval * 1000; + begin = start; + timeout = 0L; + lop = lrsample = lsample = start; + frsample = lbegin = end = Long.MAX_VALUE; + } + + private void initLimites() { + Mission mission = workerContext.getMission(); + totalOps = mission.getTotalOps() / mission.getTotalWorkers(); + totalBytes = mission.getTotalBytes() / mission.getTotalWorkers(); + if (mission.getRuntime() == 0) + return; + begin = start + mission.getRampup() * 1000; + end = begin + mission.getRuntime() * 1000; + timeout = end + mission.getRampdown() * 1000; + } + + private void initMarks() { + Set types = new LinkedHashSet(); + for (OperatorContext op : operatorRegistry) + types.add(getMarkType(op.getId(), op.getOpType(), op.getSampleType(), op.getName())); + for (String type : types) + currMarks.addMark(newMark(type)); + for (String type : types) + globalMarks.addMark(newMark(type)); + } + + private void doWork() { + doSnapshot(); + while (!workerContext.isFinished()) + try { + performOperation(); + }catch (AbortedException ae) { + if (lrsample > frsample) + doSummary(); + workerContext.setFinished(true); + } + doSnapshot(); + } + + + private void performOperation() { + if(workerContext.getAuthApi() == null || workerContext.getStorageApi() == null) + throw new AbortedException(); + if(! workerContext.getStorageApi().isAuthValid()) + reLogin(); + lbegin = System.currentTimeMillis(); + Random random = workerContext.getRandom(); + String op = operationPicker.pickOperation(random); + OperatorContext context = operatorRegistry.getOperator(op); + try{ + context.getOperator().operate(this); + }catch(AuthException ae) { + reLogin(); + } + } + + @Override + public void onSampleCreated(Sample sample) { + curr = sample.getTimestamp().getTime(); + String type = getMarkType(sample.getOpId(), sample.getOpType(), + sample.getSampleType(), sample.getOpName()); + currMarks.getMark(type).addSample(sample); + if (lbegin >= begin && lbegin < end && curr > begin && curr <= end) { + globalMarks.getMark(type).addSample(sample); + operatorRegistry.getOperator(sample.getOpId()).addSample(sample); + if (lbegin < frsample) + frsample = lbegin; // first sample emitted during runtime + lrsample = curr; // last sample collected during runtime + } + lsample = curr; // last sample collected + trySnapshot(); // make a snapshot if necessary + } + + private void trySnapshot() { + if (lsample < check) + return; + doSnapshot(); + lcheck = System.currentTimeMillis(); + check = lcheck + interval * 1000; + } + + private void doSnapshot() { + long window = lsample - lcheck; + Report report = new Report(); + for (Mark mark : currMarks) { + report.addMetrics(Metrics.convert(mark, window)); + mark.clear(); + } + Snapshot snapshot = new Snapshot(report); + workerContext.setSnapshot(snapshot); + } + + @Override + public void onOperationCompleted(Result result) { + curr = result.getTimestamp().getTime(); +/* */ + String type = getMarkType(result.getOpId(), result.getOpType(), + result.getSampleType(), result.getOpName()); + currMarks.getMark(type).addOperation(result); + if (lop >= begin && lop < end && curr > begin && curr <= end) + globalMarks.getMark(type).addOperation(result); +/* */ + lop = curr; // last operation performed + trySummary(); // make a summary report if necessary + } + + private void trySummary() { + if ((timeout <= 0 || curr < timeout) // timeout + && (totalOps <= 0 || getTotalOps() < totalOps) // operations + && (totalBytes <= 0 || getTotalBytes() < totalBytes)) // bytes + return; // not finished + doSummary(); + + workerContext.setFinished(true); + } + + private void doSummary() { +/* */ + long window = lrsample - frsample; + Report report = new Report(); + for (Mark mark : globalMarks) + report.addMetrics(Metrics.convert(mark, window)); + workerContext.setReport(report); +/* */ + } + + private int getTotalOps() { +// return ++op_count; + + int sum = 0; + for (Mark mark : globalMarks) + sum += mark.getTotalOpCount(); + return sum; + } + + private long getTotalBytes() { + long bytes = 0; + for (Mark mark : globalMarks) + bytes += mark.getByteCount(); + return bytes; + } + public void reLogin() { + LOGGER.debug("WorkAgent {} auth failed, now relogin",workerContext.getIndex()); + AuthContext authContext = workerContext.getStorageApi().getAuthContext(); + synchronized (AuthCachePool.getInstance()) { + AuthCachePool.getInstance().remove(authContext.getID()); + } + try{ + workerContext.getAuthApi().init(); + authContext = workerContext.getAuthApi().login(); + workerContext.getStorageApi().setAuthContext(authContext); + synchronized (AuthCachePool.getInstance()) { + AuthCachePool.getInstance().put(authContext.getID(), authContext); + } + LOGGER.debug("WorkAgent {} relogin successfully",workerContext.getIndex()); + }catch(AuthException ae) { + workerContext.getAuthApi().dispose(); + LOGGER.error("agent "+workerContext.getIndex()+" failed to login",ae); + throw new AgentException(); + } + } + +} diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/ConstantIntGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/ConstantIntGenerator.java index 7499e285..4fd29be1 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/ConstantIntGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/ConstantIntGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; @@ -25,9 +25,9 @@ /** * This class is to generate integers, it's for constant distribution. - * + * * @author ywang19, qzheng7 - * + * */ class ConstantIntGenerator implements IntGenerator { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/DefaultSizeGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/DefaultSizeGenerator.java index a4e78509..3b6c4efd 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/DefaultSizeGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/DefaultSizeGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; @@ -25,9 +25,9 @@ /** * This class is to generate size, which supports "10MB" like size form. - * + * * @author ywang19, qzheng7 - * + * */ class DefaultSizeGenerator implements SizeGenerator { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/Generators.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/Generators.java index 7cc18cd6..36bc9493 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/Generators.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/Generators.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/HistogramIntGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/HistogramIntGenerator.java index 80c2c106..29638445 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/HistogramIntGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/HistogramIntGenerator.java @@ -1,4 +1,22 @@ -/** +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -9,7 +27,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.driver.generator; @@ -26,111 +44,111 @@ //import com.intel.cosbench.driver.generator.RangeIntGenerator.TestThread; /** - * + * * This class provides an weighted histogram int generator. To configure it, specify a comma separated list of buckets * where each bucket is defined by a range and an integer weight. For example: - * + * * h(1|64|10,64|512|70,512|2048|20)KB - * + * * @author Christophe Vedel * */ public class HistogramIntGenerator implements IntGenerator { - private static class LowerComparator implements Comparator { - public int compare(Bucket b1, Bucket b2) { - return b1.lower - b2.lower; - } - } - - private static class Bucket { - - private final int lower; - private final UniformIntGenerator gen; - private final int weight; - private int cumulativeWeight; - - Bucket(int lower, int upper, int weight) { - this.lower = lower; - this.gen = new UniformIntGenerator(lower, upper); - this.weight = weight; - this.cumulativeWeight = 0; - } - } - - private final Bucket buckets[]; - private final int totalWeight; - - public HistogramIntGenerator(Bucket buckets[]) { - this.buckets = buckets; - totalWeight = buckets[buckets.length - 1].cumulativeWeight; - - } - - @Override - public int next(Random random) { - final int next = RandomUtils.nextInt(random, totalWeight); - for (Bucket bucket : buckets) { - if (next <= bucket.cumulativeWeight) { - return bucket.gen.next(random); - } - } - return 0; - } - - /* (non-Javadoc) - * @see com.intel.cosbench.driver.random.IntGenerator#next(java.util.Random, int, int) - */ - @Override - public int next(Random random, int idx, int all) { - throw new NotImplementedException(); - } - - public static HistogramIntGenerator parse(String pattern) { - if (!StringUtils.startsWith(pattern, "h(")) - return null; - try { - return tryParse(pattern); - } catch (Exception e) { - } - String msg = "illegal histogram distribution pattern: " + pattern; - throw new ConfigException(msg); - } - - private static HistogramIntGenerator tryParse(String pattern) { - pattern = StringUtils.substringBetween(pattern, "(", ")"); - String[] args = StringUtils.split(pattern, ','); - ArrayList bucketsList = new ArrayList(); - for (String arg : args) { - int v1 = StringUtils.indexOf(arg, '|'); - int v2 = StringUtils.lastIndexOf(arg, '|'); - boolean isOpenRange = ((v2 - v1) == 1) ? true : false; - String[] values = StringUtils.split(arg, '|'); - int lower,upper,weight; - if (isOpenRange) { - lower = Integer.parseInt(values[0]); - upper = UniformIntGenerator.getMAXupper(); - weight = Integer.parseInt(values[1]); - } else if (values.length != 3) { - throw new IllegalArgumentException(); - } else { - lower = Integer.parseInt(values[0]); - upper = Integer.parseInt(values[1]); - weight = Integer.parseInt(values[2]); - } - bucketsList.add(new Bucket(lower, upper, weight)); - } - if (bucketsList.isEmpty()) { - throw new IllegalArgumentException(); - } - Collections.sort(bucketsList, new LowerComparator()); - final Bucket[] buckets = bucketsList.toArray(new Bucket[0]); - int cumulativeWeight = 0; - for (Bucket bucket : buckets) { - cumulativeWeight += bucket.weight; - bucket.cumulativeWeight = cumulativeWeight; - } - return new HistogramIntGenerator(buckets); - } + private static class LowerComparator implements Comparator { + public int compare(Bucket b1, Bucket b2) { + return b1.lower - b2.lower; + } + } + + private static class Bucket { + + private final int lower; + private final UniformIntGenerator gen; + private final int weight; + private int cumulativeWeight; + + Bucket(int lower, int upper, int weight) { + this.lower = lower; + this.gen = new UniformIntGenerator(lower, upper); + this.weight = weight; + this.cumulativeWeight = 0; + } + } + + private final Bucket buckets[]; + private final int totalWeight; + + public HistogramIntGenerator(Bucket buckets[]) { + this.buckets = buckets; + totalWeight = buckets[buckets.length - 1].cumulativeWeight; + + } + + @Override + public int next(Random random) { + final int next = RandomUtils.nextInt(random, totalWeight); + for (Bucket bucket : buckets) { + if (next <= bucket.cumulativeWeight) { + return bucket.gen.next(random); + } + } + return 0; + } + + /* (non-Javadoc) + * @see com.intel.cosbench.driver.random.IntGenerator#next(java.util.Random, int, int) + */ + @Override + public int next(Random random, int idx, int all) { + throw new NotImplementedException(); + } + + public static HistogramIntGenerator parse(String pattern) { + if (!StringUtils.startsWith(pattern, "h(")) + return null; + try { + return tryParse(pattern); + } catch (Exception e) { + } + String msg = "illegal histogram distribution pattern: " + pattern; + throw new ConfigException(msg); + } + + private static HistogramIntGenerator tryParse(String pattern) { + pattern = StringUtils.substringBetween(pattern, "(", ")"); + String[] args = StringUtils.split(pattern, ','); + ArrayList bucketsList = new ArrayList(); + for (String arg : args) { + int v1 = StringUtils.indexOf(arg, '|'); + int v2 = StringUtils.lastIndexOf(arg, '|'); + boolean isOpenRange = ((v2 - v1) == 1) ? true : false; + String[] values = StringUtils.split(arg, '|'); + int lower,upper,weight; + if (isOpenRange) { + lower = Integer.parseInt(values[0]); + upper = UniformIntGenerator.getMAXupper(); + weight = Integer.parseInt(values[1]); + } else if (values.length != 3) { + throw new IllegalArgumentException(); + } else { + lower = Integer.parseInt(values[0]); + upper = Integer.parseInt(values[1]); + weight = Integer.parseInt(values[2]); + } + bucketsList.add(new Bucket(lower, upper, weight)); + } + if (bucketsList.isEmpty()) { + throw new IllegalArgumentException(); + } + Collections.sort(bucketsList, new LowerComparator()); + final Bucket[] buckets = bucketsList.toArray(new Bucket[0]); + int cumulativeWeight = 0; + for (Bucket bucket : buckets) { + cumulativeWeight += bucket.weight; + bucket.cumulativeWeight = cumulativeWeight; + } + return new HistogramIntGenerator(buckets); + } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/IntGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/IntGenerator.java index a2c04ef1..c1066c23 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/IntGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/IntGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/NameGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/NameGenerator.java index 8a0928a5..6de04523 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/NameGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/NameGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/NumericNameGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/NumericNameGenerator.java index 682ec3e1..54f8cb3b 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/NumericNameGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/NumericNameGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; @@ -48,11 +48,11 @@ public String next(Random random) { int value = generator.next(random); return StringUtils.join(new Object[] { prefix, value, suffix }); } - + @Override public int nextKey(Random random) { return generator.next(random); - } + } @Override public String next(Random random, int idx, int all) { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/RandomInputStream.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/RandomInputStream.java index 0ff4193f..0066939c 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/RandomInputStream.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/RandomInputStream.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; @@ -28,9 +28,9 @@ /** * This class is to generate random data as input stream for data uploading. - * + * * @author ywang19, qzheng7 - * + * */ public class RandomInputStream extends NullInputStream { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/RangeIntGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/RangeIntGenerator.java index 1ff7fbba..7d52534e 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/RangeIntGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/RangeIntGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.driver.generator; @@ -55,7 +55,7 @@ public void run() { while (i++ < 11) { int answer = generator.next(rnd, idx, all); if (!results.contains(answer)) { - System.out.println(this.getName() + ": " + answer); + System.out.println(this.getName() + ": " + answer); } else { System.out.println(this.getName() + ": " + answer + " I did get this before!"); } @@ -95,20 +95,20 @@ public RangeIntGenerator(int lower, int upper) { throw new IllegalArgumentException(); this.lower = lower; this.upper = upper; - + } private synchronized void init(int all) { - if(cursors != null) - return; - - this.cursors = new AtomicInteger[all]; - - for (int i = 0; i= idx - 1 ? idx - 1 : extra); int segment = base + (extra >= idx ? 1 : 0); - - return lower + offset + cursors[idx-1].getAndIncrement() % segment; + + return lower + offset + cursors[idx-1].getAndIncrement() % segment; } public static RangeIntGenerator parse(String pattern) { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/SequentialIntGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/SequentialIntGenerator.java index 388731bc..4f50ea9a 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/SequentialIntGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/SequentialIntGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -27,10 +27,10 @@ /** * This class supplies a thread-save int generator that returns ints from lower to upper, and then restarts at lower * again. - * - * + * + * * @author Niklas Goerke niklas974@github - * + * */ class SequentialIntGenerator implements IntGenerator { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/SizeGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/SizeGenerator.java index 3623f0e2..2a546e08 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/SizeGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/SizeGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/StreamUtils.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/StreamUtils.java index ef871985..bee2c6e5 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/StreamUtils.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/StreamUtils.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/UniformIntGenerator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/UniformIntGenerator.java index 32fe32d4..952103b6 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/UniformIntGenerator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/UniformIntGenerator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.generator; @@ -26,9 +26,9 @@ /** * This class is to generate integers, it's for uniform distribution. - * + * * @author ywang19, qzheng7 - * + * */ class UniformIntGenerator implements IntGenerator { @@ -80,7 +80,7 @@ private static UniformIntGenerator tryParse(String pattern) { } public static int getMAXupper () { - return MAXupper; - } + return MAXupper; + } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/XferCountingInputStream.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/XferCountingInputStream.java index 3dc16ee5..52ba7721 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/XferCountingInputStream.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/generator/XferCountingInputStream.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.driver.generator; import java.io.IOException; @@ -6,51 +24,51 @@ /** * This class is to record the time of data transfer - * - * + * + * */ public class XferCountingInputStream extends CountingInputStream{ - private long xferStart = 0L; - private long xferEnd = 0L; - private boolean isFirstByte = true; - - public XferCountingInputStream(InputStream in) { - super(in); - } - - @Override - public int read() throws IOException { - int result = super.read(); - recordTime(); - return result; - } - - @Override - public int read(byte[] b) throws IOException { - int result = super.read(b); - recordTime(); - return result; - } - - @Override - public int read(byte[] b, int off, int len) throws IOException { - int result = super.read(b, off, len); - recordTime(); - return result; - } - - private void recordTime() { - if (this.isFirstByte) { - this.xferStart = System.nanoTime(); - this.isFirstByte = false; - } - this.xferEnd = System.nanoTime(); - } - - public long getXferTime() { - long xferTime = (this.xferEnd - this.xferStart) / 1000000; - return xferTime > 0 ? xferTime : 0L; - } - + private long xferStart = 0L; + private long xferEnd = 0L; + private boolean isFirstByte = true; + + public XferCountingInputStream(InputStream in) { + super(in); + } + + @Override + public int read() throws IOException { + int result = super.read(); + recordTime(); + return result; + } + + @Override + public int read(byte[] b) throws IOException { + int result = super.read(b); + recordTime(); + return result; + } + + @Override + public int read(byte[] b, int off, int len) throws IOException { + int result = super.read(b, off, len); + recordTime(); + return result; + } + + private void recordTime() { + if (this.isFirstByte) { + this.xferStart = System.nanoTime(); + this.isFirstByte = false; + } + this.xferEnd = System.nanoTime(); + } + + public long getXferTime() { + long xferTime = (this.xferEnd - this.xferStart) / 1000000; + return xferTime > 0 ? xferTime : 0L; + } + } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/EmptyIterator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/EmptyIterator.java index 7aa3ff63..247884b1 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/EmptyIterator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/EmptyIterator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.iterator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/IntIterator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/IntIterator.java index b04a526d..4c268a68 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/IntIterator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/IntIterator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.iterator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/Iterators.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/Iterators.java index 0a95d077..d60ed7e5 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/Iterators.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/Iterators.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.iterator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/NameIterator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/NameIterator.java index 8dd2b063..b3030386 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/NameIterator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/NameIterator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.iterator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/NumericNameIterator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/NumericNameIterator.java index eeaec20d..00bce08a 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/NumericNameIterator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/NumericNameIterator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.iterator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/RangeIterator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/RangeIterator.java index 4273393a..9300ef08 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/RangeIterator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/iterator/RangeIterator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.iterator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/DriverContext.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/DriverContext.java index 330b58b0..44f7584e 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/DriverContext.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/DriverContext.java @@ -1,6 +1,7 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -12,32 +13,54 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.model; import java.util.HashMap; import java.util.Map; +import java.util.Date; import com.intel.cosbench.model.DriverInfo; /** * This class encapsulates the configurations in driver.conf. - * + * * @author ywang19, qzheng7 - * + * */ public class DriverContext implements DriverInfo { private String name; private String url; private boolean aliveState; + private String version; + private String time; + private String mission_dir; + // pIDMap - private Map pidMap = new HashMap(); - // logMap<'wId'+'sId', ScriptLog> - private Map scriptsLog = new HashMap(); - + private Map pidMap = new HashMap(); + // logMap<'wId'+'sId', ScriptLog> + private Map scriptsLog = new HashMap(); + + public String getTime() { + time = new Date().toString(); + return time; + } + + public void setTime(String timeStr) { + this.time = timeStr; + } + + public String getVersion() { + return version; + } + + public void setVersion(String version) { + this.version = version; + } + public DriverContext() { /* empty */ } @@ -59,36 +82,44 @@ public void setUrl(String url) { } public void setAliveState(boolean aliveState) { - this.aliveState = aliveState; - } + this.aliveState = aliveState; + } public boolean getAliveState(){ - return aliveState; - } - - public String getPidMap(String scriptName) { - String pid = pidMap.remove(scriptName); - return (pid == null) ? "0" : pid; - } - - public void putPidMap(String scriptName, String pid) { - if (pid == null) - pidMap.put(scriptName, "0"); - pidMap.put(scriptName, pid); - } - - @Override - public Map getLogMap() { - return scriptsLog; - } - - public void putLogMap(String wsId, String ScriptLog) { - if (wsId == null || wsId.isEmpty()) - return; - scriptsLog.put(wsId, ScriptLog); - } - - public String getLogMapValue(String wsId) { - return scriptsLog.remove(wsId); - } + return aliveState; + } + + public String getPidMap(String scriptName) { + String pid = pidMap.remove(scriptName); + return (pid == null) ? "0" : pid; + } + + public void putPidMap(String scriptName, String pid) { + if (pid == null) + pidMap.put(scriptName, "0"); + pidMap.put(scriptName, pid); + } + + @Override + public Map getLogMap() { + return scriptsLog; + } + + public void putLogMap(String wsId, String ScriptLog) { + if (wsId == null || wsId.isEmpty()) + return; + scriptsLog.put(wsId, ScriptLog); + } + + public String getLogMapValue(String wsId) { + return scriptsLog.remove(wsId); + } + + public String getMission_dir() { + return mission_dir; + } + + public void setMission_dir(String mission_dir) { + this.mission_dir = mission_dir; + } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/MissionContext.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/MissionContext.java index 3fa31af0..cf8bec40 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/MissionContext.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/MissionContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.model; @@ -30,9 +30,9 @@ /** * This class encapsulates behaviors of mission scheduled to current driver, one * mission is actually fragment of one workload. - * + * * @author ywang19, qzheng7 - * + * */ public class MissionContext implements MissionInfo { @@ -42,23 +42,31 @@ public class MissionContext implements MissionInfo { private StateRegistry stateHistory = new StateRegistry(); private transient XmlConfig config; private transient volatile Future future; - + private Mission mission; private LogManager logManager; private ErrorStatistics errorStatistics; private transient OperationPicker operationPicker; private transient OperatorRegistry operatorRegistry; - + private WorkerRegistry workerRegistry; /* Report will be available after the mission is accomplished */ private volatile Report report = null; // will be merged from worker reports private transient List listeners = new ArrayList(); - + private static final String GENERATE_HISTOGRAM_KEY = "histogram"; private static final boolean DEFAULT_GENERATE_HISTOGRAM = true; - + + public List getWorkerReports(){ + List wReports = new ArrayList(); + for(WorkerContext wContext:workerRegistry){ + wReports.add(wContext.getReport()); + } + return wReports; + } + public MissionContext() { errorStatistics = new ErrorStatistics(); } @@ -99,7 +107,7 @@ private void fireMissionStopped() { for (MissionListener listener : listeners) listener.missionStopped(this); } - + private Report mergeReport() { ReportMerger merger = new ReportMerger(); for (WorkerContext worker : workerRegistry) @@ -108,11 +116,11 @@ private Report mergeReport() { Config missionConfig = KVConfigParser.parse(mission.getConfig()); boolean histogram = missionConfig.getBoolean(GENERATE_HISTOGRAM_KEY, DEFAULT_GENERATE_HISTOGRAM); if(histogram) { - generateHistogram(report); + generateHistogram(report); } return report; } - + private void generateHistogram(Report report) { OperatorRegistry registry = operatorRegistry; for (Metrics metrics : report) { @@ -158,18 +166,16 @@ public LogManager getLogManager() { public void setLogManager(LogManager logManager) { this.logManager = logManager; } - - public ErrorStatistics getErrorStatistics() { - return errorStatistics; - } + return errorStatistics; + } - public void setErrorStatistics(ErrorStatistics errorStatistics) { - this.errorStatistics = errorStatistics; - } + public void setErrorStatistics(ErrorStatistics errorStatistics) { + this.errorStatistics = errorStatistics; + } - public OperationPicker getOperationPicker() { + public OperationPicker getOperationPicker() { return operationPicker; } @@ -226,16 +232,16 @@ public void addListener(MissionListener listener) { @Override public void disposeRuntime() { - if(MissionState.isStopped(state)) { - for (WorkerContext worker : workerRegistry) - worker.disposeRuntime(); - config = null; - future = null; - operationPicker = null; - operatorRegistry = null; - listeners = null; - logManager.dispose(); - } + if(MissionState.isStopped(state)) { + for (WorkerContext worker : workerRegistry) + worker.disposeRuntime(); + config = null; + future = null; + operationPicker = null; + operatorRegistry = null; + listeners = null; + logManager.dispose(); + } } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/MissionListener.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/MissionListener.java index 82c7535b..35a44d69 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/MissionListener.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/MissionListener.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.model; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/OperatorContext.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/OperatorContext.java index 92fc64ea..469415dc 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/OperatorContext.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/OperatorContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.model; @@ -24,9 +24,9 @@ /** * This class encapsulates operator related information. - * + * * @author ywang19, qzheng7 - * + * */ public class OperatorContext implements Item { @@ -43,9 +43,9 @@ public OperatorContext() { public String getName() { return operator.getName(); } - + public String getId(){ - return operator.getId(); + return operator.getId(); } public String getOpType() { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/OperatorRegistry.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/OperatorRegistry.java index 85c5ebb1..1e092b69 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/OperatorRegistry.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/OperatorRegistry.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.model; @@ -28,7 +28,7 @@ public void addOperator(OperatorContext operator) { public OperatorContext getOperator(String type) { return getItem(type); } - + public void addItem(OperatorContext item) { registry.put(item.getId(), item); } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/WorkerContext.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/WorkerContext.java index 3ce84146..94c6102a 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/model/WorkerContext.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/model/WorkerContext.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.model; @@ -30,9 +30,9 @@ /** * This class encapsulates worker related information. - * + * * @author ywang19, qzheng7 - * + * */ public class WorkerContext implements WorkerInfo { @@ -46,7 +46,7 @@ public class WorkerContext implements WorkerInfo { private volatile boolean error = false; private volatile boolean aborted = false; private volatile boolean finished = false; - + /* Each worker starts with an empty snapshot */ private transient volatile Snapshot snapshot = new Snapshot(); /* Each worker starts with an empty report */ @@ -57,7 +57,7 @@ public class WorkerContext implements WorkerInfo { private volatile int version = 0; private volatile int runlen = 0; - + public WorkerContext() { /* empty */ } @@ -121,48 +121,48 @@ public void setAborted(boolean aborted) { @Override public Snapshot getSnapshot() { - if(snapshot.getVersion() < version) - { - logger.debug("Worker[{}] : blank snapshot is generated.", index); - Snapshot blankSnapshot = new Snapshot(); - - blankSnapshot.setVersion(version); - blankSnapshot.setMinVersion(version); - blankSnapshot.setMaxVersion(version); - - version++; - runlen++; - - return blankSnapshot; - } - - // align snapshot metrics to compensate the under-counting due to blank snapshots. - if(runlen > 0) - { - Report report = snapshot.getReport(); - Metrics[] metrics = report.getAllMetrics(); - - for(int i=0; i 0) + { + Report report = snapshot.getReport(); + Metrics[] metrics = report.getAllMetrics(); + + for(int i=0; i stackTraceAndTargets = errorStatistics.getStackTraceAndTargets(); - synchronized (stackTraceAndTargets) { - if(! stackTraceAndTargets.containsKey(trace)){ - errorStatistics.getStackTraceAndException().put(trace, e); - stackTraceAndTargets.put(trace, target); - doLogErr(session.getLogger(), "worker "+ session.getIndex() + " fail to perform operation " + target, e); - } - String targets = stackTraceAndTargets.get(trace); - stackTraceAndTargets.put(trace, targets + ", "+target); - } + String trace = null; + try { + trace = e.getStackTrace()[0].toString(); + trace = e.getCause() == null ? trace : trace + e.getCause().getStackTrace()[0].toString(); + } catch (ArrayIndexOutOfBoundsException ignored) { + LOGGER.debug("Got an error with an empty stack trace. " + + "Run the driver with -XX:-OmitStackTraceInFastThrow to prevent this.", e); + trace = ""; + } + ErrorStatistics errorStatistics = session.getErrorStatistics(); + HashMap stackTraceAndTargets = errorStatistics.getStackTraceAndTargets(); + synchronized (stackTraceAndTargets) { + if(! stackTraceAndTargets.containsKey(trace)){ + errorStatistics.getStackTraceAndException().put(trace, e); + stackTraceAndTargets.put(trace, target); + doLogErr(session.getLogger(), "worker "+ session.getIndex() + " fail to perform operation " + target, e); + } + String targets = stackTraceAndTargets.get(trace); + stackTraceAndTargets.put(trace, targets + ", "+target); + } } public static void isUnauthorizedException(Exception e, Session session) { - if(e != null && e.getMessage() != null) - try{ - if(401 == Integer.valueOf(e.getMessage().substring(9, 12))){ - session.getApi().setAuthFlag(false); - LOGGER.debug("catch 401 error from storage backend, set auth flag to false"); - } - }catch(NumberFormatException ne) { - ne.printStackTrace();// mask ignore - } + if(e != null && e.getMessage() != null) + try{ + if(401 == Integer.valueOf(e.getMessage().substring(9, 12))){ + session.getApi().setAuthFlag(false); + LOGGER.debug("catch 401 error from storage backend, set auth flag to false"); + } + }catch(NumberFormatException ne) { + //ne.printStackTrace();// mask ignore + } } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Cleaner.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Cleaner.java index 713206f8..9e287d69 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Cleaner.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Cleaner.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; @@ -39,9 +39,9 @@ /** * This class encapsulates operations to delete objects, essentially, it maps to * primitive DELETE operation. - * + * * @author ywang19, qzheng7 - * + * */ class Cleaner extends AbstractOperator { @@ -94,87 +94,87 @@ protected void operate(int idx, int all, Session session) { doDispose(lastContainer, config, session); Date now = new Date(); - Result result = new Result(now, getId(), getOpType(), getSampleType(), - getName(), true); + Result result = new Result(now, getId(), getOpType(), getSampleType(), + getName(), true); session.getListener().onOperationCompleted(result); } public void doDispose(String conName, Config config, Session session) { if (Thread.interrupted()) throw new AbortedException(); - + boolean isEmpty = true; boolean tryAgain = false; do { - if(isEmpty) { - try { - session.getApi().deleteContainer(conName, config); - isEmpty = true; - tryAgain = false; - } catch (StorageInterruptedException sie) { - throw new AbortedException(); - } catch (StorageException se) { - String msg = "Error deleting container " + conName; - doLogErr(session.getLogger(), msg,se); - if(isConflictException(session,se)) { - isEmpty = false; - tryAgain = true; - }else{ - tryAgain = false; - } - } catch (Exception e) { - doLogErr(session.getLogger(), "fail to perform clean operation " + conName, e); - throw new AgentException(); // mark error - } - }else { - try{ - for(String objName: getObjectsList(conName, config, session)) { - doDelete(conName, objName, config, session, this); - } - }catch(StorageException se) { - doLogErr(session.getLogger(), "fail to get : "+conName+" objects list ",se); - tryAgain = false; - }catch (IOException ioe) { - doLogErr(session.getLogger(), "fail to convert objects stream to string",ioe); - tryAgain = false; - }catch (Exception e) { - doLogErr(session.getLogger(), "unexpected error",e); - tryAgain = false; - } - isEmpty = true; - tryAgain = true; - } + if(isEmpty) { + try { + session.getApi().deleteContainer(conName, config); + isEmpty = true; + tryAgain = false; + } catch (StorageInterruptedException sie) { + throw new AbortedException(); + } catch (StorageException se) { + String msg = "Error deleting container " + conName; + doLogErr(session.getLogger(), msg,se); + if(isConflictException(session,se)) { + isEmpty = false; + tryAgain = true; + }else{ + tryAgain = false; + } + } catch (Exception e) { + doLogErr(session.getLogger(), "fail to perform clean operation " + conName, e); + throw new AgentException(); // mark error + } + }else { + try{ + for(String objName: getObjectsList(conName, config, session)) { + doDelete(conName, objName, config, session, this); + } + }catch(StorageException se) { + doLogErr(session.getLogger(), "fail to get : "+conName+" objects list ",se); + tryAgain = false; + }catch (IOException ioe) { + doLogErr(session.getLogger(), "fail to convert objects stream to string",ioe); + tryAgain = false; + }catch (Exception e) { + doLogErr(session.getLogger(), "unexpected error",e); + tryAgain = false; + } + isEmpty = true; + tryAgain = true; + } }while(tryAgain == true); - + } - + private static String[] getObjectsList(String conName, Config config, Session session) throws IOException { - String[] objects = {}; - StringWriter stringWriter = new StringWriter(); - try { - IOUtils.copy(session.getApi().getList(conName, "", config), stringWriter); - }catch(StorageException se) { - throw se; - }catch (IOException e) { - throw e; - } - String objectString = stringWriter.toString(); - objects = objectString.split("\n"); - return objects; + String[] objects = {}; + StringWriter stringWriter = new StringWriter(); + try { + IOUtils.copy(session.getApi().getList(conName, "", config), stringWriter); + }catch(StorageException se) { + throw se; + }catch (IOException e) { + throw e; + } + String objectString = stringWriter.toString(); + objects = objectString.split("\n"); + return objects; } - + private static boolean isConflictException(Session session, Exception e) { - if(e != null && e.getMessage() != null) - try{ - if(409 == Integer.valueOf(e.getMessage().substring(9, 12))){ - doLogDebug(session.getLogger(),"catch 409 error, will clean up the unempty container and try again"); - return true; - } - }catch(NumberFormatException ne) { - ne.printStackTrace(); // mask ignore - return false; - } - return false; + if(e != null && e.getMessage() != null) + try{ + if(409 == Integer.valueOf(e.getMessage().substring(9, 12))){ + doLogDebug(session.getLogger(),"catch 409 error, will clean up the unempty container and try again"); + return true; + } + }catch(NumberFormatException ne) { + ne.printStackTrace(); // mask ignore + return false; + } + return false; } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Deleter.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Deleter.java index ea2eef61..f39bad46 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Deleter.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Deleter.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; @@ -28,9 +28,9 @@ /** * This class represents primitive DELETE operation. - * + * * @author ywang19, qzheng7 - * + * */ class Deleter extends AbstractOperator { @@ -44,7 +44,7 @@ public Deleter() { @Override protected void init(String id, int ratio, String division, Config config) { - super.init(id, ratio, division, config); + super.init(id, ratio, division, config); objPicker.init(division, config); } @@ -60,10 +60,10 @@ protected void operate(int idx, int all, Session session) { session.getListener().onSampleCreated(sample); Date now = sample.getTimestamp(); Result result = new Result(now, getId(), getOpType(), getSampleType(), - getName(), sample.isSucc()); + getName(), sample.isSucc()); session.getListener().onOperationCompleted(result); } - + public static Sample doDelete(String conName, String objName, Config config, Session session, Operator op) { if (Thread.interrupted()) @@ -77,20 +77,20 @@ public static Sample doDelete(String conName, String objName, doLogErr(session.getLogger(), sie.getMessage(), sie); throw new AbortedException(); } catch (StorageException se) { - String msg = "Error deleting object " + conName + ": " + objName; + String msg = "Error deleting object " + conName + ": " + objName; doLogWarn(session.getLogger(), msg); } catch (Exception e) { - isUnauthorizedException(e, session); - errorStatisticsHandle(e, session, conName + "/" + objName); + isUnauthorizedException(e, session); + errorStatisticsHandle(e, session, conName + "/" + objName); return new Sample(new Date(), op.getId(), op.getOpType(), - op.getSampleType(), op.getName(), false); + op.getSampleType(), op.getName(), false); } long end = System.nanoTime(); return new Sample(new Date(), op.getId(), op.getOpType(), op.getSampleType(), - op.getName(), true, (end - start) / 1000000, 0L, 0L); + op.getName(), true, (end - start) / 1000000, 0L, 0L); } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Disposer.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Disposer.java index 89f3df1c..d8be227d 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Disposer.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Disposer.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,17 +12,17 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; /** * This class encapsulates operations to delete containers, essentially, it maps * to primitive DELETE operation. - * + * * @author ywang19, qzheng7 - * + * */ public class Disposer extends Cleaner { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/FileWriter.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/FileWriter.java index e851619c..be27ef05 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/FileWriter.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/FileWriter.java @@ -1,5 +1,8 @@ -/** - +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -10,7 +13,22 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. + +*/ +/** + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ package com.intel.cosbench.driver.operator; @@ -18,6 +36,7 @@ import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; +import java.io.IOException; import java.io.InputStream; import java.security.NoSuchAlgorithmException; import java.util.Arrays; @@ -39,9 +58,9 @@ /** * This class represents a WRITE operation that reads a specified folder and writes the files contained. - * + * * @author Niklas Goerke niklas974@github - * + * */ class FileWriter extends AbstractOperator { @@ -85,20 +104,21 @@ protected void operate(int idx, int all, Session session) { Sample sample; if (!folder.canRead()) { doLogErr(session.getLogger(), "fail to perform file filewrite operation, can not read " + folder.getAbsolutePath()); - sample = new Sample(new Date(), getId(), getOpType(), - getSampleType(), getName(), false); + sample = new Sample(new Date(), getId(), getOpType(), + getSampleType(), getName(), false); } Random random = session.getRandom(); String containerName = contPicker.pickContName(random, idx, all); - + // as we index arrays starting from 0, we need to remove 1 here Integer rand = (filePicker.pickObjKey(random) - 1); String filename = null; + InputStream fis = null; + filename = listOfFiles[rand].getName(); + long length = listOfFiles[rand].length(); + try { - filename = listOfFiles[rand].getName(); - long length = listOfFiles[rand].length(); - InputStream fis = null; if (hashCheck) { HashUtil util = new HashUtil(); int hashLen = util.getHashLen(); @@ -111,23 +131,31 @@ protected void operate(int idx, int all, Session session) { sample = doWrite(fis, length, containerName, filename, config, session); } catch (FileNotFoundException e) { doLogErr(session.getLogger(), "failed to perform file Write operation, file not found", e); - sample = new Sample(new Date(), getId(), getOpType(), - getSampleType(), getName(), false); + sample = new Sample(new Date(), getId(), getOpType(), + getSampleType(), getName(), false); } catch (ArrayIndexOutOfBoundsException e) { doLogErr(session.getLogger(), "failed to perform file Write operation, tried to put more files than exist", e); sample = new Sample(new Date(), getId(), getOpType(), - getSampleType(), getName(), false); + getSampleType(), getName(), false); } catch (NoSuchAlgorithmException e) { doLogErr(session.getLogger(), "failed to perform file Write operation, hash Algorithm MD5 not supported, deaktivate hashCheck, maybe?", e); sample = new Sample(new Date(), getId(), getOpType(), - getSampleType(), getName(), false); + getSampleType(), getName(), false); + }finally { + if(fis != null) { + try { + fis.close(); + } catch (IOException e) { + doLogErr(session.getLogger(), "failed to close file " + filename, e); + } + } } session.getListener().onSampleCreated(sample); Date now = sample.getTimestamp(); - Result result = new Result(now, getId(), getOpType(), getSampleType(), - getName(), sample.isSucc()); + Result result = new Result(now, getId(), getOpType(), getSampleType(), + getName(), sample.isSucc()); session.getListener().onOperationCompleted(result); } @@ -145,10 +173,10 @@ public Sample doWrite(InputStream in, long length, String conName, String objNam doLogErr(session.getLogger(), sie.getMessage(), sie); throw new AbortedException(); } catch (Exception e) { - isUnauthorizedException(e, session); + isUnauthorizedException(e, session); doLogErr(session.getLogger(), "fail to perform filewrite operation", e); return new Sample(new Date(), getId(), getOpType(), getSampleType(), - getName(), false); + getName(), false); } finally { IOUtils.closeQuietly(cin); } @@ -156,6 +184,6 @@ public Sample doWrite(InputStream in, long length, String conName, String objNam long end = System.nanoTime(); return new Sample(new Date(), getId(), getOpType(), getSampleType(), - getName(), true, (end - start) / 1000000, cin.getXferTime(), cin.getByteCount()); + getName(), true, (end - start) / 1000000, cin.getXferTime(), cin.getByteCount()); } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Initializer.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Initializer.java index c871c9a9..51f3221c 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Initializer.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Initializer.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Lister.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Lister.java index ac42261a..d6bba42c 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Lister.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Lister.java @@ -62,8 +62,8 @@ protected void operate(int idx, int all, Session session) { Sample sample = doList(out, path[0], path[1], config, session); session.getListener().onSampleCreated(sample); Date now = sample.getTimestamp(); - Result result = new Result(now, getId(), getOpType(), getSampleType(), - getName(), sample.isSucc()); + Result result = new Result(now, getId(), getOpType(), getSampleType(), + getName(), sample.isSucc()); session.getListener().onOperationCompleted(result); } @@ -79,16 +79,16 @@ private Sample doList(OutputStream out, String conName, String objName, long xferTime = 0L; try { doLogDebug(session.getLogger(), "worker "+ session.getIndex() + " List target " + conName + "/" + objName); - in = session.getApi().getList(conName, objName, config); - long xferStart = System.nanoTime(); - copyLarge(in, cout); - xferTime = (System.nanoTime() - xferStart) / 1000000; + in = session.getApi().getList(conName, objName, config); + long xferStart = System.nanoTime(); + copyLarge(in, cout); + xferTime = (System.nanoTime() - xferStart) / 1000000; } catch (StorageInterruptedException sie) { doLogErr(session.getLogger(), sie.getMessage(), sie); throw new AbortedException(); } catch (Exception e) { - isUnauthorizedException(e, session); - errorStatisticsHandle(e, session, conName + "/" + objName); + isUnauthorizedException(e, session); + errorStatisticsHandle(e, session, conName + "/" + objName); return new Sample(new Date(), getId(), getOpType(), getSampleType(), getName(), false); } finally { @@ -97,8 +97,8 @@ private Sample doList(OutputStream out, String conName, String objName, } long end = System.nanoTime(); - return new Sample(new Date(), getId(), getOpType(), getSampleType(), - getName(), true, (end - start) / 1000000, xferTime, cout.getByteCount()); + return new Sample(new Date(), getId(), getOpType(), getSampleType(), + getName(), true, (end - start) / 1000000, xferTime, cout.getByteCount()); } public OutputStream copyLarge(InputStream input, OutputStream output) diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/OperationListener.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/OperationListener.java index bfaa5417..beee7746 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/OperationListener.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/OperationListener.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Operator.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Operator.java index 3e4b68a5..4de797c3 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Operator.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Operator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,20 +12,20 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; import com.intel.cosbench.utils.MapRegistry; public interface Operator extends MapRegistry.Item { - - public String getId(); - - public String getName(); - - public int getRatio(); + + public String getId(); + + public String getName(); + + public int getRatio(); public String getOpType(); diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Operators.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Operators.java index 37f0cf2e..5c009f9f 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Operators.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Operators.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; @@ -23,7 +23,7 @@ public class Operators { - public static Operator getOperator(Operation op, Config config) { + public static Operator getOperator(Operation op, Config config) { AbstractOperator operator = createOperator(op.getType()); operator.init(op.getId(), op.getRatio(), op.getDivision(), config); return operator; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Preparer.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Preparer.java index fdf6a28c..5c7501a8 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Preparer.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Preparer.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; @@ -36,9 +36,9 @@ /** * This class encapsulates operations to create objects, essentially, it maps to * primitive WRITE operation. - * + * * @author ywang19, qzheng7 - * + * */ class Preparer extends AbstractOperator { @@ -57,7 +57,7 @@ public Preparer() { @Override protected void init(String id, int ratio, String division, Config config) { - super.init(id, ratio, division, config); + super.init(id, ratio, division, config); objScanner.init(division, config); sizePicker.init(config); chunked = config.getBoolean("chunked", false); @@ -100,8 +100,8 @@ protected void operate(int idx, int all, Session session) { } Date now = new Date(); - Result result = new Result(now, getId(), getOpType(), getSampleType(), - getName(), true); + Result result = new Result(now, getId(), getOpType(), getSampleType(), + getName(), true); session.getListener().onOperationCompleted(result); } @@ -110,19 +110,19 @@ public static void doInit(String conName, Config config, Session session) { throw new AbortedException(); try { - session.getApi().createContainer(conName, config); + session.getApi().createContainer(conName, config); } catch (StorageInterruptedException sie) { doLogErr(session.getLogger(), sie.getMessage(), sie); throw new AbortedException(); }catch(StorageException se) { - isUnauthorizedException(se, session); + isUnauthorizedException(se, session); errorStatisticsHandle(se, session, conName); if(session.getApi().isAuthValid()){ - throw new AgentException(); + throw new AgentException(); } else { - throw new AuthException(se); - } + throw new AuthException(se); + } }catch (Exception e) { throw new AgentException(); // mark error } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Reader.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Reader.java index e0bcf285..63fc62f3 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Reader.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Reader.java @@ -69,8 +69,8 @@ protected void operate(int idx, int all, Session session) { Sample sample = doRead(out, path[0], path[1], config, session); session.getListener().onSampleCreated(sample); Date now = sample.getTimestamp(); - Result result = new Result(now, getId(), getOpType(), getSampleType(), - getName(), sample.isSucc()); + Result result = new Result(now, getId(), getOpType(), getSampleType(), + getName(), sample.isSucc()); session.getListener().onOperationCompleted(result); } @@ -90,8 +90,8 @@ private Sample doRead(OutputStream out, String conName, String objName, if (!hashCheck) { copyLarge(in, cout); } else if (!validateChecksum(conName, objName, session, in, cout)) { - return new Sample(new Date(), getId(), getOpType(), - getSampleType(), getName(), false); + return new Sample(new Date(), getId(), getOpType(), + getSampleType(), getName(), false); } long xferEnd = System.nanoTime(); xferTime = (xferEnd - xferStart) / 1000000; @@ -99,8 +99,8 @@ private Sample doRead(OutputStream out, String conName, String objName, doLogErr(session.getLogger(), sie.getMessage(), sie); throw new AbortedException(); } catch (Exception e) { - isUnauthorizedException(e, session); - errorStatisticsHandle(e, session, conName + "/" + objName); + isUnauthorizedException(e, session); + errorStatisticsHandle(e, session, conName + "/" + objName); return new Sample(new Date(), getId(), getOpType(), getSampleType(), getName(), false); } finally { @@ -109,9 +109,9 @@ private Sample doRead(OutputStream out, String conName, String objName, } long end = System.nanoTime(); - return new Sample(new Date(), getId(), getOpType(), getSampleType(), - getName(), true, (end - start)/1000000, - xferTime, cout.getByteCount()); + return new Sample(new Date(), getId(), getOpType(), getSampleType(), + getName(), true, (end - start)/1000000, + xferTime, cout.getByteCount()); } public OutputStream copyLarge(InputStream input, OutputStream output) diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Session.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Session.java index fe1a4296..d0f5ccf6 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Session.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Session.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; @@ -36,8 +36,8 @@ public interface Session { public Logger getLogger(); public OperationListener getListener(); - + public ErrorStatistics getErrorStatistics(); - + } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Writer.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Writer.java index bc70a231..1428702a 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Writer.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/operator/Writer.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.operator; @@ -35,9 +35,9 @@ /** * This class represents primitive WRITE operation. - * + * * @author ywang19, qzheng7 - * + * */ class Writer extends AbstractOperator { @@ -76,21 +76,21 @@ protected void operate(int idx, int all, Session session) { String[] path = objPicker.pickObjPath(random, idx, all); RandomInputStream in = new RandomInputStream(size, random, isRandom, hashCheck); - Sample sample = doWrite(in, len, path[0], path[1], config, session, - this); + Sample sample = doWrite(in, len, path[0], path[1], config, session, + this); session.getListener().onSampleCreated(sample); Date now = sample.getTimestamp(); - Result result = new Result(now, getId(), getOpType(), getSampleType(), - getName(), sample.isSucc()); + Result result = new Result(now, getId(), getOpType(), getSampleType(), + getName(), sample.isSucc()); session.getListener().onOperationCompleted(result); } - + public static Sample doWrite(InputStream in, long length, String conName, String objName, Config config, Session session, Operator op) { if (Thread.interrupted()) throw new AbortedException(); - - XferCountingInputStream cin = new XferCountingInputStream(in); + + XferCountingInputStream cin = new XferCountingInputStream(in); long start = System.nanoTime(); try { @@ -100,36 +100,36 @@ public static Sample doWrite(InputStream in, long length, String conName, doLogErr(session.getLogger(), sie.getMessage(), sie); throw new AbortedException(); } catch (Exception e) { - isUnauthorizedException(e, session); - errorStatisticsHandle(e, session, conName + "/" + objName); - - return new Sample(new Date(), op.getId(), op.getOpType(), - op.getSampleType(), op.getName(), false); - + isUnauthorizedException(e, session); + errorStatisticsHandle(e, session, conName + "/" + objName); + + return new Sample(new Date(), op.getId(), op.getOpType(), + op.getSampleType(), op.getName(), false); + } finally { IOUtils.closeQuietly(cin); } long end = System.nanoTime(); - return new Sample(new Date(), op.getId(), op.getOpType(), op.getSampleType(), - op.getName(), true, (end - start) / 1000000, - cin.getXferTime(), cin.getByteCount()); + return new Sample(new Date(), op.getId(), op.getOpType(), op.getSampleType(), + op.getName(), true, (end - start) / 1000000, + cin.getXferTime(), cin.getByteCount()); } /* * public static Sample doWrite(byte[] data, String conName, String objName, * Config config, Session session) { if (Thread.interrupted()) throw new * AbortedException(); - * + * * long start = System.currentTimeMillis(); - * + * * try { session.getApi().createObject(conName, objName, data, config); } * catch (StorageInterruptedException sie) { throw new AbortedException(); } * catch (Exception e) { doLog(session.getLogger(), * "fail to perform write operation", e); return new Sample(new Date(), * OP_TYPE, false); } - * + * * long end = System.currentTimeMillis(); - * + * * Date now = new Date(end); return new Sample(now, OP_TYPE, true, end - * start, data.length); } */ diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/MissionList.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/MissionList.java index df22e9c7..6f2cbf19 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/MissionList.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/MissionList.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.repository; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/MissionRepository.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/MissionRepository.java index 1ed86ed3..78448f17 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/MissionRepository.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/MissionRepository.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.repository; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/RAMMissionRepository.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/RAMMissionRepository.java index 12cb0311..fc322bb5 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/RAMMissionRepository.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/RAMMissionRepository.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.repository; @@ -28,9 +28,9 @@ /** * This class represents one in-memory repository to store all missions * information. - * + * * @author ywang19, qzheng7 - * + * */ public class RAMMissionRepository implements MissionRepository, MissionListener { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/SimpleMissionList.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/SimpleMissionList.java index bd4e5e96..6fd8dab6 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/SimpleMissionList.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/repository/SimpleMissionList.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.repository; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBAuthAPIService.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBAuthAPIService.java index 036c99fa..1ef6bc62 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBAuthAPIService.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBAuthAPIService.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.service; @@ -25,9 +25,9 @@ /** * This class is the major service for authentication on driver. - * + * * @author ywang19, qzheng7 - * + * */ public class COSBAuthAPIService implements AuthAPIService { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBDriverService.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBDriverService.java index 1e596d1c..d0f92113 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBDriverService.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBDriverService.java @@ -1,6 +1,7 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -12,8 +13,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.service; @@ -36,23 +37,23 @@ /** * This class is the major service for driver. - * + * * @author ywang19, qzheng7 - * + * */ class COSBDriverService implements DriverService, MissionListener { private static final Logger LOGGER = LogFactory.getSystemLogger(); - + private DriverContext context; private Map handlers; - + private AuthAPIService authAPIs; private StorageAPIService storageAPIs; private ExecutorService executor; private MissionRepository memRepo = new RAMMissionRepository(); - + public COSBDriverService() { /* empty */ } @@ -70,14 +71,19 @@ public void setStorageAPIs(StorageAPIService storageAPIs) { } public void init() { + if (this.context == null) { + LOGGER.error("Driver Context is not initialized."); + System.exit(-1); + } handlers = new HashMap(); handlers = Collections.synchronizedMap(handlers); executor = Executors.newCachedThreadPool(); } - + @Override - public String submit(XmlConfig config) { + public synchronized String submit(XmlConfig config) { LOGGER.debug("submitting mission ... "); + MissionContext mission = createMissionContext(config); MissionHandler handler = createHandler(mission); mission.addListener(this); @@ -104,7 +110,7 @@ private static String generateMissionId() { } private MissionHandler createHandler(MissionContext mission) { - MissionHandler handler = new MissionHandler(); + MissionHandler handler = new MissionHandler(context.getMission_dir()); handler.setMissionContext(mission); handler.setAuthAPIs(authAPIs); handler.setStorageAPIs(storageAPIs); @@ -115,6 +121,7 @@ private MissionHandler createHandler(MissionContext mission) { @Override public void login(String id) { final MissionHandler handler = handlers.get(id); + LOGGER.info("handler=" + id); if (handler == null) throw new IllegalStateException("no mission handler"); /* for strong consistency: a lock should be employed here */ @@ -130,11 +137,14 @@ public void run() { } LOGGER.debug("authing mission {} ...", id); - Future future = executor.submit(new AuthThread()); - handler.getMissionContext().setFuture(future); + Future future = null; + synchronized(handler) { + future = executor.submit(new AuthThread()); + handler.getMissionContext().setFuture(future); + } LOGGER.debug("mission {} has been requested to auth", id); + yieldExecution(200); // give mission handler a chance awaitTermination(future); // mission may be terminated or aborted - handler.getMissionContext().setFuture(null); // make sure it is null } @Override @@ -142,7 +152,6 @@ public void launch(String id) { final MissionHandler handler = handlers.get(id); if (handler == null) throw new IllegalStateException("no mission handler"); - /* for strong consistency: a lock should be employed here */ if (handler.getMissionContext().getFuture() != null) throw new IllegalStateException("mission is busy"); class DriverThread implements Runnable { @@ -155,10 +164,14 @@ public void run() { } LOGGER.debug("launching mission {} ...", id); - Future future = executor.submit(new DriverThread()); - handler.getMissionContext().setFuture(future); - yieldExecution(200); // give mission handler a chance + Future future = null; + synchronized(handler) { + future = executor.submit(new DriverThread()); + handler.getMissionContext().setFuture(future); + } + LOGGER.debug("mission {} has been requested to launch", id); + yieldExecution(200); // give mission handler a chance } @Override @@ -192,9 +205,10 @@ private static void yieldExecution(int time) { private static void awaitTermination(Future future) { try { - future.get(); // wait forever + if(future != null) + future.get(); // wait forever } catch (CancellationException ce) { - // do nothing + LOGGER.warn("task has been cancelled!", ce); } catch (InterruptedException ie) { Thread.currentThread().interrupt(); // re-interrupt } catch (Exception ee) { @@ -230,5 +244,4 @@ public void missionStopped(MissionContext mission) { handler.dispose(); LOGGER.debug("handler for mission {} has been detached", id); } - } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBDriverServiceFactory.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBDriverServiceFactory.java index 8650e416..a8a50939 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBDriverServiceFactory.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBDriverServiceFactory.java @@ -1,6 +1,7 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -12,14 +13,17 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.service; import static org.apache.commons.lang.SystemUtils.IS_OS_WINDOWS; +import java.io.BufferedReader; import java.io.File; +import java.io.FileReader; +import java.io.IOException; import com.intel.cosbench.api.auth.AuthAPIService; import com.intel.cosbench.api.storage.StorageAPIService; @@ -86,9 +90,36 @@ private DriverContext getDriverContext() { DriverContext context = new DriverContext(); context.setName(loadDriverName()); context.setUrl(loadDriverUrl()); + context.setVersion(getVersion()); + context.setMission_dir(loadMissionDir()); return context; } + private String getVersion() { + // TODO Auto-generated method stub + String str = getName("VERSION"); + String str2 = getName("BUILD.no"); + return str+"."+str2; + } + + private String getName(String fileName){ + String str = null ; + File myFile=new File(fileName); + if(!myFile.exists()){ + System.err.println("Can't Find " + fileName); + } + try { + BufferedReader in = new BufferedReader(new FileReader(myFile)); + str = in.readLine(); + in.close(); + } + catch (IOException e) { + e.getStackTrace(); + } + return str; + } + + protected String loadLogLevel() { return config.get("driver.log_level", "INFO"); } @@ -105,4 +136,7 @@ private String loadDriverUrl() { return config.get("driver.url", "N/A"); } + private String loadMissionDir() { + return config.get("driver.mission_dir", "log/mission"); + } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBStorageAPIService.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBStorageAPIService.java index 64fba1fb..a8d85f28 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBStorageAPIService.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/COSBStorageAPIService.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.service; @@ -25,9 +25,9 @@ /** * This class is the major service for storage on driver. - * + * * @author ywang19, qzheng7 - * + * */ public class COSBStorageAPIService implements StorageAPIService { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/MissionException.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/MissionException.java index af5b26d4..4f039360 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/MissionException.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/MissionException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.service; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/MissionHandler.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/MissionHandler.java index 5f2afb48..fe08062d 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/service/MissionHandler.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/service/MissionHandler.java @@ -1,6 +1,7 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -12,8 +13,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.service; @@ -48,12 +49,7 @@ class MissionHandler { private static final Logger LOGGER = LogFactory.getSystemLogger(); - private static final File LOG_DIR = new File(new File("log"), "mission"); - - static { - if (!LOG_DIR.exists()) - LOG_DIR.mkdirs(); - } + private static File LOG_DIR; private int retry; // auth retry number private Config authConfig; // shared auth configurations @@ -65,8 +61,9 @@ class MissionHandler { private ExecutorService executor; private MissionContext missionContext; - public MissionHandler() { - /* empty */ + public MissionHandler(String mission_dir) { + LOG_DIR = new File(mission_dir); + if (!LOG_DIR.exists()) LOG_DIR.mkdirs(); } public MissionContext getMissionContext() { @@ -137,34 +134,34 @@ private static OperatorContext createOperatorContext(Operation op) { context.setOperator(Operators.getOperator(op, config)); return context; } - - private void initOpDefaultName(Mission mission) { - Set opTypes = new HashSet(); - for (Operation op : mission.getOperations()) { - opTypes.add(op.getType()); - } - if (opTypes.size() == mission.getOperations().size()) - return; - for (String opType : opTypes) { - int index = 0; - for (Operation op : mission.getOperations()) { - if (op.getType().equals(opType)) { - index++; - if (op.getConfig().indexOf("name") < 0) { - op.setConfig(StringUtils.join(new Object[] { - op.getConfig(), ";name=", - String.valueOf(index), "-", op.getType() })); - } - } - } - } - } + + private void initOpDefaultName(Mission mission) { + Set opTypes = new HashSet(); + for (Operation op : mission.getOperations()) { + opTypes.add(op.getType()); + } + if (opTypes.size() == mission.getOperations().size()) + return; + for (String opType : opTypes) { + int index = 0; + for (Operation op : mission.getOperations()) { + if (op.getType().equals(opType)) { + index++; + if (op.getConfig().indexOf("name") < 0) { + op.setConfig(StringUtils.join(new Object[] { + op.getConfig(), ";name=", + String.valueOf(index), "-", op.getType() })); + } + } + } + } + } private void initOpPicker() { OperationPicker picker = new OperationPicker(); Mission mission = missionContext.getMission(); for (Operation op : mission) - picker.addOperation(op.getId(), op.getRatio()); + picker.addOperation(op.getId(), op.getRatio()); missionContext.setOperationPicker(picker); } @@ -245,21 +242,15 @@ public void login() { private void performLogin() { missionContext.setState(AUTHING); -// // Use worker 0 for authentication (and use the same token for other workers) -// WorkerContext worker0 = missionContext.getWorkerRegistry().getWorkerByIndex(0); -// List agents = createAuthAgentFromContext(worker0); List agents = createAuthAgents(); executeAgents(agents, 0); -// AuthContext authContext = worker0.getStorageApi().getAuthContext(); -// setAllWorkersAuthContext(authContext); - missionContext.setState(AUTHED); } @SuppressWarnings("unused") private void setAllWorkersAuthContext(AuthContext authContext) { - for (WorkerContext workerContext : missionContext.getWorkerRegistry()) + for (WorkerContext workerContext : missionContext.getWorkerRegistry()) workerContext.getStorageApi().setAuthContext(authContext); } @@ -274,7 +265,7 @@ private List createAuthAgentFromContext(WorkerContext workerContext) { return agents; } - private List createAuthAgents() { + private List createAuthAgents() { List agents = new ArrayList(); for (WorkerContext workerContext : missionContext.getWorkerRegistry()) agents.add(Agents.newAuthAgent(retry, workerContext)); @@ -313,7 +304,7 @@ public void stress() { return; } LOGGER.info("mission {} has been executed successfully", id); - + } private void stressTarget() { @@ -367,19 +358,19 @@ public void close() { "mission should be in the state of finished"); String id = missionContext.getId(); missionContext.setState(ACCOMPLISHED); - for (int i = 0; i < missionContext.getReport().getAllMetrics().length; i++) { - LOGGER.debug("!!!! mission op: " - + missionContext.getReport().getAllMetrics()[i].getOpType() - + "-" - + missionContext.getReport().getAllMetrics()[i].getOpType()); - if (missionContext.getReport().getAllMetrics()[i].getSampleCount() == 0 - && missionContext.getReport().getAllMetrics()[i] - .getTotalSampleCount() > 0) { - missionContext.setState(FAILED); - LOGGER.debug("!!!! mission opt -> FAILED"); - break; - } - } + for (int i = 0; i < missionContext.getReport().getAllMetrics().length; i++) { + LOGGER.debug("!!!! mission op: " + + missionContext.getReport().getAllMetrics()[i].getOpType() + + "-" + + missionContext.getReport().getAllMetrics()[i].getOpType()); + if (missionContext.getReport().getAllMetrics()[i].getSampleCount() == 0 + && missionContext.getReport().getAllMetrics()[i] + .getTotalSampleCount() > 0) { + missionContext.setState(FAILED); + LOGGER.debug("!!!! mission opt -> FAILED"); + break; + } + } LOGGER.info("mission {} has been closed successfully", id); } @@ -403,27 +394,27 @@ public void abort() { private void abortAgents(boolean shutdownNow) { Thread.interrupted(); // clear interruption status - - executor.shutdown(); - try { - // Wait a few seconds for existing tasks to terminate - if (!executor.awaitTermination(5, TimeUnit.SECONDS)) { - executor.shutdownNow(); - - String id = missionContext.getId(); - - if (!awaitTermination(5) && !awaitTermination(10) && !awaitTermination(30)) - LOGGER.warn("fail to abort agents for mission {}", id); - else - LOGGER.info("all agents have been aborted in mission {}", id); - - LOGGER.info("mission {} appears to be aborted", id); // agents aborted - } - - } catch (InterruptedException ie) { - executor.shutdownNow(); - Thread.currentThread().interrupt(); - } + + executor.shutdown(); + try { + // Wait a few seconds for existing tasks to terminate + if (!executor.awaitTermination(5, TimeUnit.SECONDS)) { + executor.shutdownNow(); + + String id = missionContext.getId(); + + if (!awaitTermination(5) && !awaitTermination(10) && !awaitTermination(30)) + LOGGER.warn("fail to abort agents for mission {}", id); + else + LOGGER.info("all agents have been aborted in mission {}", id); + + LOGGER.info("mission {} appears to be aborted", id); // agents aborted + } + + } catch (InterruptedException ie) { + executor.shutdownNow(); + Thread.currentThread().interrupt(); + } } private boolean awaitTermination(int seconds) { @@ -437,5 +428,4 @@ private boolean awaitTermination(int seconds) { } return executor.isTerminated(); } - } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/AuthCachePool.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/AuthCachePool.java index a21bbe05..d6bc6e05 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/AuthCachePool.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/AuthCachePool.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.driver.util; import java.util.concurrent.*; @@ -5,20 +23,20 @@ /** * This class encapsulates an auth cache pool to help cache authenticated AuthContext for reuse. - * + * * @author ywang19 - * + * */ public class AuthCachePool extends ConcurrentHashMap{ - private static final long serialVersionUID = -8592973423618299263L; - private static AuthCachePool INSTANCE = new AuthCachePool(); - - private AuthCachePool() { - - } - - public static AuthCachePool getInstance() { - return INSTANCE; - } + private static final long serialVersionUID = -8592973423618299263L; + private static AuthCachePool INSTANCE = new AuthCachePool(); + + private AuthCachePool() { + + } + + public static AuthCachePool getInstance() { + return INSTANCE; + } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ContainerPicker.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ContainerPicker.java index 17d27932..a6600420 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ContainerPicker.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ContainerPicker.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.driver.util; @@ -29,9 +29,9 @@ /** * This class encapsulates logic to pick names for the container - * + * * @author Niklas Goerke - niklas974@github - * + * */ public class ContainerPicker { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/Defaults.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/Defaults.java index 1e6fcbbf..599de3b3 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/Defaults.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/Defaults.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.util; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/Division.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/Division.java index 869b0b1b..47b403f5 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/Division.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/Division.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.util; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/FilePicker.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/FilePicker.java index 64bc799a..2d3f64a1 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/FilePicker.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/FilePicker.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.driver.util; @@ -25,9 +25,9 @@ /** * This class encapsulates logic to pick integers for identififaction of files / objects - * + * * @author Niklas Goerke - niklas974@github - * + * */ public class FilePicker { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/HashUtil.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/HashUtil.java index afc0a539..901e050d 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/HashUtil.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/HashUtil.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.util; diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/HashedFileInputStream.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/HashedFileInputStream.java index 28000efa..3beca593 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/HashedFileInputStream.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/HashedFileInputStream.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,7 +12,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.driver.util; @@ -26,9 +26,9 @@ /** * This class is used to concatinate the file given with its hash (hash is formatted !!!!$hash!!!!) - * + * * @author Niklas Goerke niklas974@github - * + * */ public class HashedFileInputStream extends NullInputStream { @@ -53,7 +53,7 @@ public HashedFileInputStream(File file, boolean hashCheck, HashUtil hashutil, lo this.size = size; this.hashLen = this.util.getHashLen(); } - + @Override protected int processByte() { throw new UnsupportedOperationException("do not read byte by byte"); diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ObjectPicker.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ObjectPicker.java index 52e1fe8b..6e45e153 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ObjectPicker.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ObjectPicker.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.util; @@ -28,9 +28,9 @@ /** * This class encapsulates logic to pick up objects. - * + * * @author ywang19, qzheng7 - * + * */ public class ObjectPicker { @@ -48,18 +48,18 @@ public void init(String division, Config config) { objNmGen = getObjNmGen(config, Boolean.FALSE); this.division = Division.getDivision(division); } - + public void init4Lister(String division, Config config) { conNmGen = getConNmGen(config, Boolean.TRUE); objNmGen = getObjNmGen(config, Boolean.TRUE); this.division = Division.getDivision(division); - } + } private static NameGenerator getConNmGen(Config config, boolean isLister) { String pattern = isLister ? config.get("containers", null) - : config.get("containers"); + : config.get("containers"); if (pattern == null) - return null; + return null; String prefix = config.get("cprefix", CONTAINER_PREFIX); String suffix = config.get("csuffix", CONTAINER_SUFFIX); return Generators.getNameGenerator(pattern, prefix, suffix); @@ -67,16 +67,16 @@ private static NameGenerator getConNmGen(Config config, boolean isLister) { private static NameGenerator getObjNmGen(Config config, boolean isLister) { String pattern = isLister ? config.get("objects", null) - : config.get("objects"); + : config.get("objects"); if (pattern == null) - return null; + return null; String prefix = config.get("oprefix", OBJECT_PREFIX); String suffix = config.get("osuffix", OBJECT_SUFFIX); return Generators.getNameGenerator(pattern, prefix, suffix); } public String[] pickObjPath(Random random, int idx, int all) { - synchronized(this) { + synchronized(this) { if (division.equals(OBJECT)) return new String[] { conNmGen.next(random), objNmGen.next(random, idx, all) }; @@ -84,22 +84,22 @@ public String[] pickObjPath(Random random, int idx, int all) { return new String[] { conNmGen.next(random, idx, all), objNmGen.next(random) }; return new String[] { conNmGen.next(random), objNmGen.next(random) }; - } + } } - + /* a path picker for Lister */ public String[] pickTargetPath(Random random, int idx, int all) { - synchronized (this) { - if (conNmGen == null && objNmGen != null) { - throw new ConfigException("no such key defined: " + "containers"); - } else if (conNmGen == null && objNmGen == null) { - return new String[] { "", "" }; - } else if (objNmGen == null) { - return new String[] { conNmGen.next(random, idx, all), "" }; - } else { - return new String[] { conNmGen.next(random), objNmGen.next(random) }; - } - } - } + synchronized (this) { + if (conNmGen == null && objNmGen != null) { + throw new ConfigException("no such key defined: " + "containers"); + } else if (conNmGen == null && objNmGen == null) { + return new String[] { "", "" }; + } else if (objNmGen == null) { + return new String[] { conNmGen.next(random, idx, all), "" }; + } else { + return new String[] { conNmGen.next(random), objNmGen.next(random) }; + } + } + } } diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ObjectScanner.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ObjectScanner.java index 913762a9..eec5e55f 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ObjectScanner.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/ObjectScanner.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.util; @@ -25,9 +25,9 @@ /** * This class encapsulates logic to scan object ranges. - * + * * @author ywang19, qzheng7 - * + * */ public class ObjectScanner { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/OperationPicker.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/OperationPicker.java index 7608a81e..99ee7011 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/OperationPicker.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/OperationPicker.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.util; @@ -23,9 +23,9 @@ /** * This class encapsulates logic to pick up operations. - * + * * @author ywang19, qzheng7 - * + * */ public class OperationPicker { diff --git a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/SizePicker.java b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/SizePicker.java index 8eda239b..0fafc57d 100644 --- a/dev/cosbench-driver/src/com/intel/cosbench/driver/util/SizePicker.java +++ b/dev/cosbench-driver/src/com/intel/cosbench/driver/util/SizePicker.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.driver.util; @@ -24,9 +24,9 @@ /** * This class encapsulates logic to pick up object sizes. - * + * * @author ywang19, qzheng7 - * + * */ public class SizePicker { diff --git a/dev/cosbench-ecs/.classpath b/dev/cosbench-ecs/.classpath new file mode 100644 index 00000000..dbdfc0ce --- /dev/null +++ b/dev/cosbench-ecs/.classpath @@ -0,0 +1,22 @@ + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/cosbench-ecs/.project b/dev/cosbench-ecs/.project new file mode 100644 index 00000000..e4f0aac4 --- /dev/null +++ b/dev/cosbench-ecs/.project @@ -0,0 +1,28 @@ + + + cosbench-ecs + + + + + + org.eclipse.jdt.core.javabuilder + + + + + org.eclipse.pde.ManifestBuilder + + + + + org.eclipse.pde.SchemaBuilder + + + + + + org.eclipse.pde.PluginNature + org.eclipse.jdt.core.javanature + + diff --git a/dev/cosbench-ecs/META-INF/.ecsmeta/.dirmeta b/dev/cosbench-ecs/META-INF/.ecsmeta/.dirmeta deleted file mode 100644 index 170229a4..00000000 --- a/dev/cosbench-ecs/META-INF/.ecsmeta/.dirmeta +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":0,"contentType":"application/x-directory","httpExpires":null,"modificationTime":"Jun 23, 2016 4:03:00 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2016-06-20T21:16:47Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-23T21:02:46Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2016-06-20T21:16:47Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rwxr-xr-x","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/META-INF/.ecsmeta/MANIFEST.MF b/dev/cosbench-ecs/META-INF/.ecsmeta/MANIFEST.MF deleted file mode 100644 index 6d1e583e..00000000 --- a/dev/cosbench-ecs/META-INF/.ecsmeta/MANIFEST.MF +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":777,"contentType":"application/octet-stream","httpExpires":null,"modificationTime":"Jun 23, 2016 4:03:49 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2016-06-20T21:16:47Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-20T21:16:47Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2016-06-20T21:16:47Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rw-r--r--","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/META-INF/MANIFEST.MF b/dev/cosbench-ecs/META-INF/MANIFEST.MF index 476b4b12..1d0726fa 100644 --- a/dev/cosbench-ecs/META-INF/MANIFEST.MF +++ b/dev/cosbench-ecs/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: cosbench-ecs Bundle-SymbolicName: cosbench-ecs -Bundle-Version: 0.4.2.0 +Bundle-Version: 0.4.7 Bundle-Vendor: emc Import-Package: com.intel.cosbench.api.auth, com.intel.cosbench.api.context, diff --git a/dev/cosbench-ecs/src/.ecsmeta/.dirmeta b/dev/cosbench-ecs/src/.ecsmeta/.dirmeta deleted file mode 100644 index 51f85ea1..00000000 --- a/dev/cosbench-ecs/src/.ecsmeta/.dirmeta +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":0,"contentType":"application/x-directory","httpExpires":null,"modificationTime":"Jun 23, 2016 4:03:00 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2014-11-19T20:38:22Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-23T21:02:46Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2014-11-19T20:38:22Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rwxr-xr-x","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/.ecsmeta/.dirmeta b/dev/cosbench-ecs/src/com/.ecsmeta/.dirmeta deleted file mode 100644 index 40e820d0..00000000 --- a/dev/cosbench-ecs/src/com/.ecsmeta/.dirmeta +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":0,"contentType":"application/x-directory","httpExpires":null,"modificationTime":"Jun 23, 2016 4:03:50 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2014-11-19T20:38:22Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-23T21:02:46Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2014-11-19T20:38:22Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rwxr-xr-x","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/emc/.ecsmeta/.dirmeta b/dev/cosbench-ecs/src/com/emc/.ecsmeta/.dirmeta deleted file mode 100644 index c931195c..00000000 --- a/dev/cosbench-ecs/src/com/emc/.ecsmeta/.dirmeta +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":0,"contentType":"application/x-directory","httpExpires":null,"modificationTime":"Jun 23, 2016 4:06:27 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2015-03-06T18:29:54Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-23T21:02:47Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2015-03-06T18:29:54Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rwxr-xr-x","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/emc/vipr/.ecsmeta/.dirmeta b/dev/cosbench-ecs/src/com/emc/vipr/.ecsmeta/.dirmeta deleted file mode 100644 index bea511d9..00000000 --- a/dev/cosbench-ecs/src/com/emc/vipr/.ecsmeta/.dirmeta +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":0,"contentType":"application/x-directory","httpExpires":null,"modificationTime":"Jun 23, 2016 4:11:56 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2014-11-19T20:38:22Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-23T21:02:47Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2014-11-19T20:38:22Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rwxr-xr-x","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/.ecsmeta/.dirmeta b/dev/cosbench-ecs/src/com/emc/vipr/cosbench/.ecsmeta/.dirmeta deleted file mode 100644 index 0e619466..00000000 --- a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/.ecsmeta/.dirmeta +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":0,"contentType":"application/x-directory","httpExpires":null,"modificationTime":"Jun 23, 2016 4:18:41 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2015-03-11T16:53:12Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-23T21:02:47Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2015-03-11T16:53:12Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rwxr-xr-x","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/.dirmeta b/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/.dirmeta deleted file mode 100644 index 65793fef..00000000 --- a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/.dirmeta +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":0,"contentType":"application/x-directory","httpExpires":null,"modificationTime":"Jun 23, 2016 4:23:56 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2016-06-20T15:52:33Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-23T21:02:48Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2016-06-20T15:52:33Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rwxr-xr-x","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSConstants.java b/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSConstants.java deleted file mode 100644 index 1c74319f..00000000 --- a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSConstants.java +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":2529,"contentType":"application/octet-stream","httpExpires":null,"modificationTime":"Jun 23, 2016 4:30:40 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2016-05-24T15:54:45Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-20T15:23:45Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2016-05-24T15:54:45Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rw-r--r--","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSStorage.java b/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSStorage.java deleted file mode 100644 index 4699571b..00000000 --- a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSStorage.java +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":10046,"contentType":"application/octet-stream","httpExpires":null,"modificationTime":"Jun 23, 2016 4:30:40 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2016-06-13T22:12:05Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-16T16:58:53Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2016-06-13T22:12:05Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rw-r--r--","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSStorageFactory.java b/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSStorageFactory.java deleted file mode 100644 index 738bd2fd..00000000 --- a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/.ecsmeta/ECSStorageFactory.java +++ /dev/null @@ -1 +0,0 @@ -{"instanceClass":"com.emc.ecs.sync.model.SyncMetadata","cacheControl":null,"contentDisposition":null,"contentEncoding":"identity","contentLength":1150,"contentType":"application/octet-stream","httpExpires":null,"modificationTime":"Jun 23, 2016 4:30:40 PM","userMetadata":{"x-emc-crtime":{"key":"x-emc-crtime","value":"2015-05-15T18:49:33Z","indexed":false},"x-emc-posix-owner-name":{"key":"x-emc-posix-owner-name","value":"petera5","indexed":false},"x-emc-posix-group-owner-name":{"key":"x-emc-posix-group-owner-name","value":"CORP\\Domain Users","indexed":false},"x-emc-atime":{"key":"x-emc-atime","value":"2016-06-16T16:58:53Z","indexed":false},"x-emc-mtime":{"key":"x-emc-mtime","value":"2015-05-15T18:49:33Z","indexed":false},"x-emc-posix-mode":{"key":"x-emc-posix-mode","value":"rw-r--r--","indexed":false}},"acl":null,"checksum":null,"expirationDate":null} \ No newline at end of file diff --git a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/ECSConstants.java b/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/ECSConstants.java index c2ea2e8f..0fb44cff 100644 --- a/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/ECSConstants.java +++ b/dev/cosbench-ecs/src/com/emc/vipr/cosbench/ECSStor/ECSConstants.java @@ -18,7 +18,7 @@ /** * Constants for use with EMC ECS Storage API plugin * for COSBench. - * + * * The EMC ECS plugin requires at least access key and * secret key to be user-defined. If unspecified, ECS * endpoint will default to s3.amazonaws.com with a connection @@ -32,26 +32,26 @@ public interface ECSConstants { // Read Timeout Control; default is same as connection timeout public static final String READ_TIMEOUT_KEY = "read_timeout"; - + // ECS Endpoint Constants public static final String ENDPOINT_KEY = "endpoint"; public static final String ENDPOINT_DEFAULT = "http://s3.amazonaws.com"; - + // ECS Access Key public static final String AUTH_USERNAME_KEY = "accesskey"; public static final String AUTH_USERNAME_DEFAULT = ""; - + // ECS Secret Key public static final String AUTH_PASSWORD_KEY = "secretkey"; public static final String AUTH_PASSWORD_DEFAULT = ""; - + // Proxy Port/Host Keys - currently unsupported // public static final String PROXY_HOST_KEY = "proxyhost"; // public static final String PROXY_PORT_KEY = "proxyport"; - + // Namespace Key public static final String NAMESPACE_KEY = "namespace"; - + // Path Style Access Constants public static final String PATH_STYLE_ACCESS_KEY = "path_style_access"; public static final boolean PATH_STYLE_ACCESS_DEFAULT = false; diff --git a/dev/cosbench-gcs/META-INF/MANIFEST.MF b/dev/cosbench-gcs/META-INF/MANIFEST.MF index 558f8d85..77dbafc6 100644 --- a/dev/cosbench-gcs/META-INF/MANIFEST.MF +++ b/dev/cosbench-gcs/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Google Cloud Storage Client Bundle Bundle-SymbolicName: cosbench-gcs -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: Intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-gcs/src/com/intel/cosbench/api/GcsStor/GcsStorage.java b/dev/cosbench-gcs/src/com/intel/cosbench/api/GcsStor/GcsStorage.java index 5d61edda..18992f6c 100644 --- a/dev/cosbench-gcs/src/com/intel/cosbench/api/GcsStor/GcsStorage.java +++ b/dev/cosbench-gcs/src/com/intel/cosbench/api/GcsStor/GcsStorage.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.GcsStor; import static com.intel.cosbench.client.GcsStor.GcsStorConstants.CONN_TIMEOUT_KEY; @@ -25,123 +43,123 @@ import com.google.api.services.storage.model.StorageObject; public class GcsStorage extends NoneStorage { - private int timeout; - private String jsonKeyFile; - private String projectId; - private static Storage client; - - private static final JsonFactory JSON_FACTORY = JacksonFactory - .getDefaultInstance(); - - @Override - public void init(Config config, Logger logger) { - super.init(config, logger); - initParms(config); - try { - InputStream input = new FileInputStream(jsonKeyFile); - GoogleCredential credential = GoogleCredential.fromStream(input); - if (credential.createScopedRequired()) { - credential = credential.createScoped(StorageScopes.all()); - } - HttpTransport httpTransport = GoogleNetHttpTransport - .newTrustedTransport(); - client = new Storage.Builder(httpTransport, JSON_FACTORY, - credential).build(); - logger.debug("GCS client has been initialized"); - } catch (Exception e) { - throw new StorageException(e); - } - } - - private void initParms(Config config) { - timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); - parms.put(CONN_TIMEOUT_KEY, timeout); - - jsonKeyFile = config.get(JSON_KEY_FILE, JSON_KEY_FILE_DEFAULT); - parms.put(JSON_KEY_FILE, jsonKeyFile); - - projectId = config.get(PROJECT_ID, PROJECT_ID_DEFAULT); - parms.put(PROJECT_ID, projectId); - - logger.debug("using storage config: {}", parms); - } - - @Override - public void setAuthContext(AuthContext info) { - super.setAuthContext(info); - } - - @Override - public void dispose() { - super.dispose(); - client = null; - } - - @Override - public InputStream getObject(String container, String object, Config config) { - super.getObject(container, object, config); - InputStream stream; - try { - Storage.Objects.Get getRequest = client.objects().get(container, - object); - stream = getRequest.executeMediaAsInputStream(); - } catch (IOException e) { - throw new StorageException(e); - } - return stream; - } - - @Override - public void createContainer(String container, Config config) { - super.createContainer(container, config); - Bucket newBucket = new Bucket(); - newBucket.setName(container); - try { - Storage.Buckets.Insert bucketInsertRequest = client.buckets() - .insert(projectId, newBucket); - bucketInsertRequest.execute(); - } catch (IOException e) { - throw new StorageException(e); - } - } - - @Override - public void createObject(String container, String object, InputStream data, - long length, Config config) { - super.createObject(container, object, data, length, config); - InputStreamContent contentStream = new InputStreamContent( - "application/octet-stream", data); - StorageObject objectMetadata = new StorageObject().setName(object); - try { - Storage.Objects.Insert insertRequest = client.objects().insert( - container, objectMetadata, contentStream); - insertRequest.execute(); - } catch (IOException e) { - throw new StorageException(e); - } - - } - - @Override - public void deleteContainer(String container, Config config) { - super.deleteContainer(container, config); - try { - Storage.Buckets.Delete bucketDeleteRequest = client.buckets() - .delete(container); - bucketDeleteRequest.execute(); - } catch (IOException e) { - throw new StorageException(e); - } - } - - @Override - public void deleteObject(String container, String object, Config config) { - super.deleteObject(container, object, config); - try { - client.objects().delete(container, object).execute(); - } catch (Exception e) { - throw new StorageException(e); - } - } + private int timeout; + private String jsonKeyFile; + private String projectId; + private static Storage client; + + private static final JsonFactory JSON_FACTORY = JacksonFactory + .getDefaultInstance(); + + @Override + public void init(Config config, Logger logger) { + super.init(config, logger); + initParms(config); + try { + InputStream input = new FileInputStream(jsonKeyFile); + GoogleCredential credential = GoogleCredential.fromStream(input); + if (credential.createScopedRequired()) { + credential = credential.createScoped(StorageScopes.all()); + } + HttpTransport httpTransport = GoogleNetHttpTransport + .newTrustedTransport(); + client = new Storage.Builder(httpTransport, JSON_FACTORY, + credential).build(); + logger.debug("GCS client has been initialized"); + } catch (Exception e) { + throw new StorageException(e); + } + } + + private void initParms(Config config) { + timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); + parms.put(CONN_TIMEOUT_KEY, timeout); + + jsonKeyFile = config.get(JSON_KEY_FILE, JSON_KEY_FILE_DEFAULT); + parms.put(JSON_KEY_FILE, jsonKeyFile); + + projectId = config.get(PROJECT_ID, PROJECT_ID_DEFAULT); + parms.put(PROJECT_ID, projectId); + + logger.debug("using storage config: {}", parms); + } + + @Override + public void setAuthContext(AuthContext info) { + super.setAuthContext(info); + } + + @Override + public void dispose() { + super.dispose(); + client = null; + } + + @Override + public InputStream getObject(String container, String object, Config config) { + super.getObject(container, object, config); + InputStream stream; + try { + Storage.Objects.Get getRequest = client.objects().get(container, + object); + stream = getRequest.executeMediaAsInputStream(); + } catch (IOException e) { + throw new StorageException(e); + } + return stream; + } + + @Override + public void createContainer(String container, Config config) { + super.createContainer(container, config); + Bucket newBucket = new Bucket(); + newBucket.setName(container); + try { + Storage.Buckets.Insert bucketInsertRequest = client.buckets() + .insert(projectId, newBucket); + bucketInsertRequest.execute(); + } catch (IOException e) { + throw new StorageException(e); + } + } + + @Override + public void createObject(String container, String object, InputStream data, + long length, Config config) { + super.createObject(container, object, data, length, config); + InputStreamContent contentStream = new InputStreamContent( + "application/octet-stream", data); + StorageObject objectMetadata = new StorageObject().setName(object); + try { + Storage.Objects.Insert insertRequest = client.objects().insert( + container, objectMetadata, contentStream); + insertRequest.execute(); + } catch (IOException e) { + throw new StorageException(e); + } + + } + + @Override + public void deleteContainer(String container, Config config) { + super.deleteContainer(container, config); + try { + Storage.Buckets.Delete bucketDeleteRequest = client.buckets() + .delete(container); + bucketDeleteRequest.execute(); + } catch (IOException e) { + throw new StorageException(e); + } + } + + @Override + public void deleteObject(String container, String object, Config config) { + super.deleteObject(container, object, config); + try { + client.objects().delete(container, object).execute(); + } catch (Exception e) { + throw new StorageException(e); + } + } } diff --git a/dev/cosbench-gcs/src/com/intel/cosbench/api/GcsStor/GcsStorageFactory.java b/dev/cosbench-gcs/src/com/intel/cosbench/api/GcsStor/GcsStorageFactory.java index 000d3e78..61b2c911 100644 --- a/dev/cosbench-gcs/src/com/intel/cosbench/api/GcsStor/GcsStorageFactory.java +++ b/dev/cosbench-gcs/src/com/intel/cosbench/api/GcsStor/GcsStorageFactory.java @@ -1,17 +1,35 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.GcsStor; import com.intel.cosbench.api.storage.*; public class GcsStorageFactory implements StorageAPIFactory { - @Override - public String getStorageName() { - return "gcs"; - } + @Override + public String getStorageName() { + return "gcs"; + } - @Override - public StorageAPI getStorageAPI() { - return new GcsStorage(); - } + @Override + public StorageAPI getStorageAPI() { + return new GcsStorage(); + } } diff --git a/dev/cosbench-gcs/src/com/intel/cosbench/client/GcsStor/GcsStorConstants.java b/dev/cosbench-gcs/src/com/intel/cosbench/client/GcsStor/GcsStorConstants.java index 954d41e7..b009f77d 100644 --- a/dev/cosbench-gcs/src/com/intel/cosbench/client/GcsStor/GcsStorConstants.java +++ b/dev/cosbench-gcs/src/com/intel/cosbench/client/GcsStor/GcsStorConstants.java @@ -1,24 +1,42 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.client.GcsStor; public interface GcsStorConstants { - // -------------------------------------------------------------------------- - // CONNECTION - // -------------------------------------------------------------------------- + // -------------------------------------------------------------------------- + // CONNECTION + // -------------------------------------------------------------------------- - String CONN_TIMEOUT_KEY = "timeout"; - int CONN_TIMEOUT_DEFAULT = 30000; + String CONN_TIMEOUT_KEY = "timeout"; + int CONN_TIMEOUT_DEFAULT = 30000; - // -------------------------------------------------------------------------- - // PROJECT_ID - // -------------------------------------------------------------------------- - String PROJECT_ID = "projectid"; - String PROJECT_ID_DEFAULT = "xxxxx-xxxx"; + // -------------------------------------------------------------------------- + // PROJECT_ID + // -------------------------------------------------------------------------- + String PROJECT_ID = "projectid"; + String PROJECT_ID_DEFAULT = "xxxxx-xxxx"; - // -------------------------------------------------------------------------- - // JSON_KEY_FILE - // -------------------------------------------------------------------------- - String JSON_KEY_FILE = "jsonkeyfile"; - String JSON_KEY_FILE_DEFAULT = "/path/to/json/key/file"; + // -------------------------------------------------------------------------- + // JSON_KEY_FILE + // -------------------------------------------------------------------------- + String JSON_KEY_FILE = "jsonkeyfile"; + String JSON_KEY_FILE_DEFAULT = "/path/to/json/key/file"; } diff --git a/dev/cosbench-http/META-INF/MANIFEST.MF b/dev/cosbench-http/META-INF/MANIFEST.MF index 057d2f7b..aab0aaae 100644 --- a/dev/cosbench-http/META-INF/MANIFEST.MF +++ b/dev/cosbench-http/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Http Client Utility Bundle Bundle-SymbolicName: cosbench-http -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Export-Package: com.intel.cosbench.client.http @@ -19,6 +19,7 @@ Import-Package: org.apache.commons.codec;version="[1.3.0,2.0.0)", org.apache.http.entity;version="[4.1.4,5.0.0)", org.apache.http.impl.client;version="[4.1.3,5.0.0)", org.apache.http.impl.conn;version="[4.1.3,5.0.0)", + org.apache.http.impl.conn.tsccm;version="4.1.3", org.apache.http.message;version="[4.1.4,5.0.0)", org.apache.http.params;version="[4.1.4,5.0.0)", org.apache.http.protocol;version="[4.1.4,5.0.0)", diff --git a/dev/cosbench-http/src/com/intel/cosbench/client/http/HttpClientException.java b/dev/cosbench-http/src/com/intel/cosbench/client/http/HttpClientException.java index 9d4c4ea6..c75d0514 100644 --- a/dev/cosbench-http/src/com/intel/cosbench/client/http/HttpClientException.java +++ b/dev/cosbench-http/src/com/intel/cosbench/client/http/HttpClientException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.http; @@ -25,9 +25,9 @@ public class HttpClientException extends HttpResponseException { private StatusLine httpStatusLine; public HttpClientException(int code, String message) { - super(code, message); + super(code, message); } - + public HttpClientException(int code, String message, Header[] httpHeaders, StatusLine httpStatusLine) { super(code, message); @@ -42,7 +42,7 @@ public Header[] getHttpHeaders() { public StatusLine getHttpStatusLine() { return httpStatusLine; } - + private static final long serialVersionUID = 1L; } diff --git a/dev/cosbench-http/src/com/intel/cosbench/client/http/HttpClientUtil.java b/dev/cosbench-http/src/com/intel/cosbench/client/http/HttpClientUtil.java index 78657383..5e3c50a8 100644 --- a/dev/cosbench-http/src/com/intel/cosbench/client/http/HttpClientUtil.java +++ b/dev/cosbench-http/src/com/intel/cosbench/client/http/HttpClientUtil.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.http; @@ -43,6 +43,7 @@ import org.apache.http.conn.ssl.SSLSocketFactory; import org.apache.http.impl.client.DefaultHttpClient; import org.apache.http.impl.conn.SingleClientConnManager; +import org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager; import org.apache.http.params.BasicHttpParams; import org.apache.http.params.HttpConnectionParams; import org.apache.http.params.HttpParams; @@ -53,101 +54,87 @@ /** * This class encapsulates basic HTTP client related functions which are * necessary for REST based storage system. - * + * * @author ywang19, qzheng7 - * + * */ public class HttpClientUtil { /** * Creates a default HTTP client with a given timeout setting.
- * Note that this client is NOT Thread-Safe and can only be - * used by a single thread. - * + * * @param timeout * the timeout in seconds that will be honored by this client * @return a new HTTP client */ public static HttpClient createHttpClient(int timeout) { - // make it support self-signed certification for https. - HttpParams params = createDefaultHttpParams(timeout); - ClientConnectionManager cm = createClientConnManager(); - - return new DefaultHttpClient(cm, params); - } - - private static HttpParams createDefaultHttpParams(int timeout) { + // make it support self-signed certification for https. HttpParams params = new BasicHttpParams(); /* default HTTP parameters */ DefaultHttpClient.setDefaultHttpParams(params); /* connection/socket timeouts */ HttpConnectionParams.setSoTimeout(params, timeout); HttpConnectionParams.setConnectionTimeout(params, timeout); - /* user agent */ HttpProtocolParams.setUserAgent(params, "cosbench/2.0"); - return params; + + SchemeRegistry sr = new SchemeRegistry(); + sr.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory())); + sr.register(new Scheme("https", 443, createSSLSocketFactory())); + ClientConnectionManager cm = new ThreadSafeClientConnManager(sr); + + return new DefaultHttpClient(cm, params); } - + @SuppressWarnings({ "deprecation"}) - private static SSLSocketFactory createSSLSocketFactory() + private static SSLSocketFactory createSSLSocketFactory() { - try - { - SSLContext ctx = SSLContext.getInstance("TLS"); - X509TrustManager tm = new X509TrustManager() { - @Override - public void checkClientTrusted(X509Certificate[] xcs, String string) throws CertificateException { - } - @Override - public void checkServerTrusted(X509Certificate[] xcs, String string) throws CertificateException { - } - - public X509Certificate[] getAcceptedIssuers() { - return null; - } - }; - ctx.init(null, new X509TrustManager[]{tm}, null); - String[] enabled = {"SSL_RSA_WITH_NULL_MD5","SSL_RSA_WITH_NULL_SHA"}; - ctx.createSSLEngine().setEnabledCipherSuites(enabled); - - SSLSocketFactory ssf = new SSLSocketFactory(ctx); - ssf.setHostnameVerifier(SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER); - + try + { + SSLContext ctx = SSLContext.getInstance("TLS"); + X509TrustManager tm = new X509TrustManager() { + @Override + public void checkClientTrusted(X509Certificate[] xcs, String string) throws CertificateException { + } + @Override + public void checkServerTrusted(X509Certificate[] xcs, String string) throws CertificateException { + } + + public X509Certificate[] getAcceptedIssuers() { + return null; + } + }; + ctx.init(null, new X509TrustManager[]{tm}, null); + String[] enabled = {"SSL_RSA_WITH_NULL_MD5","SSL_RSA_WITH_NULL_SHA"}; + ctx.createSSLEngine().setEnabledCipherSuites(enabled); + + SSLSocketFactory ssf = new SSLSocketFactory(ctx); + ssf.setHostnameVerifier(SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER); + return ssf; - } catch (Exception ex) { - ex.printStackTrace(); - return null; - } + } catch (Exception ex) { + ex.printStackTrace(); + return null; + } } - private static ClientConnectionManager createClientConnManager() - { - SchemeRegistry sr = new SchemeRegistry(); - - sr.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory())); - sr.register(new Scheme("https", 443, createSSLSocketFactory())); - - return new SingleClientConnManager(sr); - } - /** * Releases the resources held by the given HTTP client.
* Note that no further connections can be made upon a disposed HTTP client. - * + * * @param client * the HTTP client to be disposed. */ public static void disposeHttpClient(HttpClient client) { - if(client != null) { - ClientConnectionManager manager = client.getConnectionManager(); - manager.shutdown(); - client = null; - } + if(client != null) { + ClientConnectionManager manager = client.getConnectionManager(); + manager.shutdown(); + client = null; + } } public static HttpGet makeHttpGet(String url) { - return new HttpGet(url); + return new HttpGet(url); } public static HttpPut makeHttpPut(String url) { diff --git a/dev/cosbench-httpauth/META-INF/MANIFEST.MF b/dev/cosbench-httpauth/META-INF/MANIFEST.MF index 8cac2177..ee14d4f0 100644 --- a/dev/cosbench-httpauth/META-INF/MANIFEST.MF +++ b/dev/cosbench-httpauth/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Http Authentication Bundle Bundle-SymbolicName: cosbench-httpauth -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuth.java b/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuth.java index 0da2b328..4194b5ee 100644 --- a/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuth.java +++ b/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuth.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.httpauth; @@ -54,13 +54,13 @@ /** * This class encapsulates a Http BASIC/DIGEST Authentication implementation for the Auth-API. - * + * * @author ywang19 - * + * */ class HttpAuth extends NoneAuth { - private AbstractHttpClient client = null; - + private AbstractHttpClient client = null; + /* account info */ private String auth_url; private String username; @@ -72,17 +72,17 @@ class HttpAuth extends NoneAuth { public HttpAuth() { /* empty */ } - + public HttpAuth(String url, String username, String password, int timeout) { -// this.host = host; -// this.port = port; - this.auth_url = url; - this.username = username; - this.password = password; - this.timeout = timeout; - - this.client = new DefaultHttpClient(); - this.client.getParams().setIntParameter(CoreConnectionPNames.SO_TIMEOUT, timeout); +// this.host = host; +// this.port = port; + this.auth_url = url; + this.username = username; + this.password = password; + this.timeout = timeout; + + this.client = new DefaultHttpClient(); + this.client.getParams().setIntParameter(CoreConnectionPNames.SO_TIMEOUT, timeout); } @Override @@ -94,16 +94,16 @@ public void init(Config config, Logger logger) { // port = config.getInt(AUTH_PORT_KEY, AUTH_PORT_DEFAULT); auth_url = config.get(AUTH_URL_KEY, AUTH_URL_DEFAULT); username = config.get(AUTH_USERNAME_KEY, AUTH_USERNAME_DEFAULT); - password = config.get(AUTH_PASSWORD_KEY, AUTH_PASSWORD_DEFAULT); + password = config.get(AUTH_PASSWORD_KEY, AUTH_PASSWORD_DEFAULT); timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); logger.debug("using auth config: {}", parms); client = (DefaultHttpClient)HttpClientUtil.createHttpClient(timeout); - + logger.debug("httpauth client has been initialized"); } - + @Override public AuthContext login() { super.login(); @@ -111,38 +111,38 @@ public AuthContext login() { // HttpHost host = new HttpHost(); // HttpHost targetHost = new HttpHost(host, port, protocol); - - URI uri; - - try { - uri = new URI(auth_url); - }catch(URISyntaxException use) { - throw new AuthException(use); - } - - HttpGet method = new HttpGet(auth_url); - method.getParams().setIntParameter(CoreConnectionPNames.SO_TIMEOUT, timeout); - - client.getCredentialsProvider().setCredentials(new AuthScope(uri.getHost(), uri.getPort()), - new UsernamePasswordCredentials(this.username, this.password)); - - HttpContext localContext = new BasicHttpContext(); - localContext.setAttribute(AuthPNames.TARGET_AUTH_PREF, Arrays.asList(new String[] {AuthPolicy.BASIC, AuthPolicy.DIGEST})); - - HttpResponse response = null; - - try { - dumpClientSettings(); - response = client.execute(method, localContext); - - dumpClientSettings(); - if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) { - return createContext(); - } - if (response.getStatusLine().getStatusCode() == HttpStatus.SC_UNAUTHORIZED) { - throw new AuthException(response.getStatusLine().getReasonPhrase()); - } - }catch (SocketTimeoutException ste) { + + URI uri; + + try { + uri = new URI(auth_url); + }catch(URISyntaxException use) { + throw new AuthException(use); + } + + HttpGet method = new HttpGet(auth_url); + method.getParams().setIntParameter(CoreConnectionPNames.SO_TIMEOUT, timeout); + + client.getCredentialsProvider().setCredentials(new AuthScope(uri.getHost(), uri.getPort()), + new UsernamePasswordCredentials(this.username, this.password)); + + HttpContext localContext = new BasicHttpContext(); + localContext.setAttribute(AuthPNames.TARGET_AUTH_PREF, Arrays.asList(new String[] {AuthPolicy.BASIC, AuthPolicy.DIGEST})); + + HttpResponse response = null; + + try { + dumpClientSettings(); + response = client.execute(method, localContext); + + dumpClientSettings(); + if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) { + return createContext(); + } + if (response.getStatusLine().getStatusCode() == HttpStatus.SC_UNAUTHORIZED) { + throw new AuthException(response.getStatusLine().getReasonPhrase()); + } + }catch (SocketTimeoutException ste) { throw new AuthTimeoutException(ste); } catch (ConnectTimeoutException cte) { throw new AuthTimeoutException(cte); @@ -150,61 +150,61 @@ public AuthContext login() { throw new AuthInterruptedException(ie); } catch (Exception e) { throw new AuthException(e); - } finally { - if(response != null) - try { - dumpResponse(response); - EntityUtils.consume(response.getEntity()); - }catch(Exception ignore) { - ignore.printStackTrace(); - } - - if (method != null) - method.abort(); - } - + } finally { + if(response != null) + try { + dumpResponse(response); + EntityUtils.consume(response.getEntity()); + }catch(Exception ignore) { + ignore.printStackTrace(); + } + + if (method != null) + method.abort(); + } + return createContext(); } - - + + private AuthContext createContext() { // AuthContext context = new DefaultAuthContext(); // context.put(AUTH_CLIENT_KEY, client); // context.put(STORAGE_URL_KEY, auth_url); -// +// // return context; HttpAuthContext context = new HttpAuthContext(auth_url, username, password, client); - + return context; } - + private void dumpClientSettings() { - System.out.println(client.getAuthSchemes().getSchemeNames()); - System.out.println(client.getCredentialsProvider()); - + System.out.println(client.getAuthSchemes().getSchemeNames()); + System.out.println(client.getCredentialsProvider()); + } private void dumpResponse(HttpResponse response) { - try { - System.out.println("\nStatus Line"); - System.out.println("-----------"); - System.out.println(response.getStatusLine()); - - Header authHeader = response.getFirstHeader(AUTH.WWW_AUTH); - System.out.println("Auth Header = " + authHeader); - Header authRspHeader = response.getFirstHeader(AUTH.WWW_AUTH_RESP); - System.out.println("Auth Rsp Header = " + authRspHeader); - System.out.println("\nHeaders"); - System.out.println("-------"); - for(Header header : response.getAllHeaders()) { - System.out.println(header.toString()); - } - - System.out.println("\nBody"); - System.out.println("----"); - System.out.println(EntityUtils.toString(response.getEntity())); - EntityUtils.consume(response.getEntity()); - }catch(Exception e) { - e.printStackTrace(); - } + try { + System.out.println("\nStatus Line"); + System.out.println("-----------"); + System.out.println(response.getStatusLine()); + + Header authHeader = response.getFirstHeader(AUTH.WWW_AUTH); + System.out.println("Auth Header = " + authHeader); + Header authRspHeader = response.getFirstHeader(AUTH.WWW_AUTH_RESP); + System.out.println("Auth Rsp Header = " + authRspHeader); + System.out.println("\nHeaders"); + System.out.println("-------"); + for(Header header : response.getAllHeaders()) { + System.out.println(header.toString()); + } + + System.out.println("\nBody"); + System.out.println("----"); + System.out.println(EntityUtils.toString(response.getEntity())); + EntityUtils.consume(response.getEntity()); + }catch(Exception e) { + e.printStackTrace(); + } } } diff --git a/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthContext.java b/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthContext.java index 9948b494..b9e8b5f0 100644 --- a/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthContext.java +++ b/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthContext.java @@ -1,69 +1,69 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.api.httpauth; - -import static com.intel.cosbench.client.httpauth.HttpAuthConstants.*; - -import org.apache.http.client.HttpClient; - -import com.intel.cosbench.api.context.AuthContext; - -/** - * This class encapsulates a HttpAuth specific AuthContext. - * - * @author ywang19 - * - */ -public class HttpAuthContext extends AuthContext { - - public HttpAuthContext(String auth_url, String username, String password, HttpClient client) { - this.put(AUTH_URL_KEY, auth_url); - this.put(AUTH_USERNAME_KEY, username); - this.put(AUTH_PASSWORD_KEY, password); - this.put(AUTH_CLIENT_KEY, client); - } - - @Override - public String getID(String meta) { - StringBuilder builder = new StringBuilder(); - builder.append(getID()); - builder.append(";meta="); - builder.append(meta); - - return builder.toString(); - } - - @Override - public String getID() { - StringBuilder builder = new StringBuilder(); - builder.append(AUTH_URL_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_URL_KEY)); - builder.append(";"); - builder.append(AUTH_USERNAME_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_USERNAME_KEY)); - builder.append(";"); - builder.append(AUTH_PASSWORD_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_PASSWORD_KEY)); - - return builder.toString(); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.api.httpauth; + +import static com.intel.cosbench.client.httpauth.HttpAuthConstants.*; + +import org.apache.http.client.HttpClient; + +import com.intel.cosbench.api.context.AuthContext; + +/** + * This class encapsulates a HttpAuth specific AuthContext. + * + * @author ywang19 + * + */ +public class HttpAuthContext extends AuthContext { + + public HttpAuthContext(String auth_url, String username, String password, HttpClient client) { + this.put(AUTH_URL_KEY, auth_url); + this.put(AUTH_USERNAME_KEY, username); + this.put(AUTH_PASSWORD_KEY, password); + this.put(AUTH_CLIENT_KEY, client); + } + + @Override + public String getID(String meta) { + StringBuilder builder = new StringBuilder(); + builder.append(getID()); + builder.append(";meta="); + builder.append(meta); + + return builder.toString(); + } + + @Override + public String getID() { + StringBuilder builder = new StringBuilder(); + builder.append(AUTH_URL_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_URL_KEY)); + builder.append(";"); + builder.append(AUTH_USERNAME_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_USERNAME_KEY)); + builder.append(";"); + builder.append(AUTH_PASSWORD_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_PASSWORD_KEY)); + + return builder.toString(); + } + +} diff --git a/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthFactory.java b/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthFactory.java index 89f70376..a39b8054 100644 --- a/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthFactory.java +++ b/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.httpauth; diff --git a/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthTest.java b/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthTest.java index 99518827..72ddd4e3 100644 --- a/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthTest.java +++ b/dev/cosbench-httpauth/src/com/intel/cosbench/api/httpauth/HttpAuthTest.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.httpauth; @@ -31,78 +31,78 @@ class HttpAuthTest { - private AbstractHttpClient client = null; - + private AbstractHttpClient client = null; + /* account info */ private String storage_url; public boolean getObject(AuthContext context, String conName, String objName) { -// host = context.getStr(AUTH_HOST_KEY); -// port = context.getInt(AUTH_PORT_KEY); - storage_url = context.getStr(STORAGE_URL_KEY); - client = (AbstractHttpClient)context.get(AUTH_CLIENT_KEY); - - +// host = context.getStr(AUTH_HOST_KEY); +// port = context.getInt(AUTH_PORT_KEY); + storage_url = context.getStr(STORAGE_URL_KEY); + client = (AbstractHttpClient)context.get(AUTH_CLIENT_KEY); + + String url = storage_url + "/" + conName + "/" + objName; - HttpGet method = new HttpGet(url); - - HttpResponse response = null; - - try { - response = client.execute(method); - - if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) { - return true; - } - if (response.getStatusLine().getStatusCode() == HttpStatus.SC_UNAUTHORIZED) { - System.out.println("unauthorized request!"); - return false; - } - } catch(Exception e) { - e.printStackTrace(); - } finally { - if(response != null) - dumpResponse(response); - - if (method != null) - method.abort(); - } - + HttpGet method = new HttpGet(url); + + HttpResponse response = null; + + try { + response = client.execute(method); + + if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) { + return true; + } + if (response.getStatusLine().getStatusCode() == HttpStatus.SC_UNAUTHORIZED) { + System.out.println("unauthorized request!"); + return false; + } + } catch(Exception e) { + e.printStackTrace(); + } finally { + if(response != null) + dumpResponse(response); + + if (method != null) + method.abort(); + } + return false; } - + private void dumpResponse(HttpResponse response) { - try { - System.out.println("\nStatus Line"); - System.out.println("-----------"); - System.out.println(response.getStatusLine()); - - Header authHeader = response.getFirstHeader(AUTH.WWW_AUTH); - System.out.println("Auth Header = " + authHeader); - Header authRspHeader = response.getFirstHeader(AUTH.WWW_AUTH_RESP); - System.out.println("Auth Rsp Header = " + authRspHeader); - System.out.println("\nHeaders"); - System.out.println("-------"); - for(Header header : response.getAllHeaders()) { - System.out.println(header.toString()); - } - - System.out.println("\nBody"); - System.out.println("----"); - System.out.println(EntityUtils.toString(response.getEntity())); - EntityUtils.consume(response.getEntity()); - }catch(Exception e) { - e.printStackTrace(); - } + try { + System.out.println("\nStatus Line"); + System.out.println("-----------"); + System.out.println(response.getStatusLine()); + + Header authHeader = response.getFirstHeader(AUTH.WWW_AUTH); + System.out.println("Auth Header = " + authHeader); + Header authRspHeader = response.getFirstHeader(AUTH.WWW_AUTH_RESP); + System.out.println("Auth Rsp Header = " + authRspHeader); + System.out.println("\nHeaders"); + System.out.println("-------"); + for(Header header : response.getAllHeaders()) { + System.out.println(header.toString()); + } + + System.out.println("\nBody"); + System.out.println("----"); + System.out.println(EntityUtils.toString(response.getEntity())); + EntityUtils.consume(response.getEntity()); + }catch(Exception e) { + e.printStackTrace(); + } } - + public static void main(String[] args) { - HttpAuthTest test = new HttpAuthTest(); - - HttpAuth auth = new HttpAuth("http://localhost:8080", "cdmi", "mypass", 3000); - - test.getObject(auth.login(), "cdmi-test", "abc.txt"); + HttpAuthTest test = new HttpAuthTest(); + + HttpAuth auth = new HttpAuth("http://localhost:8080", "cdmi", "mypass", 3000); + + test.getObject(auth.login(), "cdmi-test", "abc.txt"); } } diff --git a/dev/cosbench-httpauth/src/com/intel/cosbench/client/httpauth/HttpAuthConstants.java b/dev/cosbench-httpauth/src/com/intel/cosbench/client/httpauth/HttpAuthConstants.java index 3465eccb..13134f30 100644 --- a/dev/cosbench-httpauth/src/com/intel/cosbench/client/httpauth/HttpAuthConstants.java +++ b/dev/cosbench-httpauth/src/com/intel/cosbench/client/httpauth/HttpAuthConstants.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.httpauth; diff --git a/dev/cosbench-keystone/META-INF/MANIFEST.MF b/dev/cosbench-keystone/META-INF/MANIFEST.MF index 784d2b8f..54a36586 100644 --- a/dev/cosbench-keystone/META-INF/MANIFEST.MF +++ b/dev/cosbench-keystone/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Keystone Client Bundle Bundle-SymbolicName: cosbench-keystone -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuth.java b/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuth.java index f4b2c801..9b90c7a9 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuth.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuth.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.keystone; @@ -31,9 +31,9 @@ /** * This class encapsulates an Openstack Keystone implementation for the * Auth-API. - * + * * @author ywang19, qzheng7 - * + * */ class KeystoneAuth extends NoneAuth { @@ -49,12 +49,17 @@ class KeystoneAuth extends NoneAuth { private String tenantId; private String tenantName; + /*keystone region*/ + private String region; + /* service info */ private String service; /* connection setting */ private int timeout; + Logger logger = null; + public KeystoneAuth() { /* empty */ } @@ -62,7 +67,7 @@ public KeystoneAuth() { @Override public void init(Config config, Logger logger) { super.init(config, logger); - + this.logger = logger; url = config.get(AUTH_URL_KEY, config.get(AUTH_URL_ALTKEY, URL_DEFAULT)); username = config.get(AUTH_USERNAME_KEY, AUTH_USERNAME_DEFAULT); password = config.get(AUTH_PASSWORD_KEY, AUTH_PASSWORD_DEFAULT); @@ -71,6 +76,7 @@ public void init(Config config, Logger logger) { tenantName = config.get(AUTH_TENANT_NAME_KEY, config.get(AUTH_TENANT_NAME_ALTKEY, AUTH_TENANT_NAME_DEFAULT)); service = config.get(AUTH_SERVICE_KEY, AUTH_SERVICE_DEFAULT); timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); + region = config.get(AUTH_REGION_KEY, AUTH_REGION_DEFAULT); parms.put(AUTH_URL_KEY, url); parms.put(AUTH_USERNAME_KEY, username); @@ -80,11 +86,13 @@ public void init(Config config, Logger logger) { parms.put(AUTH_TENANT_NAME_KEY, tenantName); parms.put(AUTH_SERVICE_KEY, service); parms.put(CONN_TIMEOUT_KEY, timeout); + parms.put(AUTH_REGION_KEY, AUTH_REGION_DEFAULT); + logger.debug("using auth config: {}", parms); HttpClient httpClient = HttpClientUtil.createHttpClient(timeout); - client = new KeystoneClient(httpClient, url, username, password, + client = new KeystoneClient(logger, httpClient, url, username, password, tenantName, timeout); logger.debug("keystone client has been initialized"); } @@ -117,8 +125,8 @@ private AuthContext createContext() { // context.put(AUTH_TOKEN_KEY, client.getKeystoneTokenId()); // context.put(STORAGE_URL_KEY, client.getServiceUrl(service)); // return context; - KeystoneAuthContext context = new KeystoneAuthContext(url, username, password, service, client.getKeystoneTokenId(), client.getServiceUrl(service)); - + KeystoneAuthContext context = new KeystoneAuthContext(url, username, password, service, client.getKeystoneTokenId(), client.getServiceUrl(service,region)); + return context; } diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuthContext.java b/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuthContext.java index e48e3056..0b92ccf1 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuthContext.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuthContext.java @@ -1,68 +1,68 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.api.keystone; - -import static com.intel.cosbench.client.keystone.KeystoneConstants.*; -import com.intel.cosbench.api.context.AuthContext; - -/** - * This class encapsulates a Keystone specific AuthContext. - * - * @author ywang19 - * - */ -public class KeystoneAuthContext extends AuthContext { - - public KeystoneAuthContext(String url, String username, String password, String service, String token, String storage_url) { - this.put(AUTH_URL_KEY, url); - this.put(AUTH_USERNAME_KEY, username); - this.put(AUTH_PASSWORD_KEY, password); - this.put(AUTH_SERVICE_KEY, service); - this.put(AUTH_TOKEN_KEY, token); - this.put(STORAGE_URL_KEY, storage_url); - } - - @Override - public String getID(String meta) { - StringBuilder builder = new StringBuilder(); - builder.append(getID()); - builder.append(";meta="); - builder.append(meta); - - return builder.toString(); - } - - @Override - public String getID() { - StringBuilder builder = new StringBuilder(); - builder.append(AUTH_URL_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_URL_KEY)); - builder.append(";"); - builder.append(AUTH_USERNAME_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_USERNAME_KEY)); - builder.append(";"); - builder.append(AUTH_PASSWORD_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_PASSWORD_KEY)); - - return builder.toString(); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.api.keystone; + +import static com.intel.cosbench.client.keystone.KeystoneConstants.*; +import com.intel.cosbench.api.context.AuthContext; + +/** + * This class encapsulates a Keystone specific AuthContext. + * + * @author ywang19 + * + */ +public class KeystoneAuthContext extends AuthContext { + + public KeystoneAuthContext(String url, String username, String password, String service, String token, String storage_url) { + this.put(AUTH_URL_KEY, url); + this.put(AUTH_USERNAME_KEY, username); + this.put(AUTH_PASSWORD_KEY, password); + this.put(AUTH_SERVICE_KEY, service); + this.put(AUTH_TOKEN_KEY, token); + this.put(STORAGE_URL_KEY, storage_url); + } + + @Override + public String getID(String meta) { + StringBuilder builder = new StringBuilder(); + builder.append(getID()); + builder.append(";meta="); + builder.append(meta); + + return builder.toString(); + } + + @Override + public String getID() { + StringBuilder builder = new StringBuilder(); + builder.append(AUTH_URL_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_URL_KEY)); + builder.append(";"); + builder.append(AUTH_USERNAME_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_USERNAME_KEY)); + builder.append(";"); + builder.append(AUTH_PASSWORD_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_PASSWORD_KEY)); + + return builder.toString(); + } + +} diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuthFactory.java b/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuthFactory.java index 7fda998b..5543b07e 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuthFactory.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/api/keystone/KeystoneAuthFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.keystone; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneAuthException.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneAuthException.java index 909f1e48..71951326 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneAuthException.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneAuthException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneClient.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneClient.java index a7435227..aeed8118 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneClient.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneClient.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,11 +12,12 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; +import java.util.ArrayList; import java.util.List; import org.apache.http.client.HttpClient; @@ -27,13 +28,14 @@ import com.intel.cosbench.client.keystone.KeystoneResponse.AccessInfo.Token; import com.intel.cosbench.client.keystone.KeystoneResponse.AccessInfo.User; import com.intel.cosbench.client.keystone.handler.*; +import com.intel.cosbench.log.Logger; /** * A client for Openstack keystone authentication service.
* The client can be used to obtain a valid keystone token in exchange of * correct credentials. According to keystone, there are different ways to * achieve this: - * + * *
    *
  1. Simple Authentication
    2. *

    @@ -53,11 +55,13 @@ * order to get a new token that will be scoped with the specified tenant. *

    *
- * - * @author qzheng (qing.zheng@intel.com) + * @author ywang19 + * @author qzheng */ public class KeystoneClient { + private Logger logger; + /* user info */ private String username; private String password; @@ -67,14 +71,17 @@ public class KeystoneClient { private String tenantId; private String tenantName; + /*targe region*/ + private String region; /* authentication handler */ private AuthHandler handler; /* authentication response */ private KeystoneResponse response; - public KeystoneClient(HttpClient client, String url, String username, + public KeystoneClient(Logger logger, HttpClient client, String url, String username, String password, String tenantName, int timeout) { + this.logger = logger; this.username = username; this.password = password; this.tenantName = tenantName; @@ -85,7 +92,7 @@ public KeystoneClient(HttpClient client, String url, String username, * Perform the authentication against a keystone service. Once the * authentication is successfully completed, the results can be retrieved * from the client instance. - * + * * @see KeystoneClient#getKeystoneTokenId() * @see KeystoneClient#getServiceUrl(String) */ @@ -270,7 +277,7 @@ public void setHandler(AuthHandler handler) { * only be called after the client has successfully performed an * authentication against the keystone service, null will be * returned otherwise. - * + * * @return the keystone token id */ public String getKeystoneTokenId() { @@ -282,18 +289,43 @@ public String getKeystoneTokenId() { * This method should only be called after the client has successfully * performed an authentication against the keystone service, * null will be returned otherwise. - * + * * @param serviceName * - the name identifying the service * @return the public URL of a cloud service */ - public String getServiceUrl(String serviceName) { + public String getServiceUrl(String serviceName, String region) { ServiceInfo service = getServiceInfo(serviceName); if (service == null) return null; List endpoints = service.getEndpoints(); - if (endpoints != null && endpoints.size() > 0) - return endpoints.get(0).getPublicURL(); + + if (endpoints == null || endpoints.size() == 0) + { + logger.error("no endpoints return from keystone"); + return null; + } + + List regions = new ArrayList(); + for (Endpoint endpoint : endpoints) { + String the_region = endpoint.getRegion(); + if(the_region != null) { + regions.add(the_region); + } + } + + if (region == null || region.isEmpty()) { // no region assigned, will use the first one. + + logger.warn("Below regions are returned from keystone : " + regions.toString() + + ", but no expected region assigned in your configuration, so the first region will be used."); + return endpoints.get(0).getPublicURL(); + } + + int idx = -1; + if((idx=regions.indexOf(region)) >= 0) { + return endpoints.get(idx).getPublicURL(); + } + return null; } @@ -302,7 +334,7 @@ public String getServiceUrl(String serviceName) { * given name. This method should only be called after the client has * successfully performed an authentication against the keystone service, * null will be returned otherwise. - * + * * @param serviceName * - the name identifying the service * @return the information regarding a cloud service @@ -314,6 +346,12 @@ public ServiceInfo getServiceInfo(String serviceName) { : service.getName() == null) return service; + List services = new ArrayList(); + for (ServiceInfo service : catalog) + services.add(service.getName()); + + logger.error("no designated service [" + serviceName + "] found, but only those services returned: " + services.toString()); + return null; } diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneClientException.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneClientException.java index 56d116b7..225c8d3e 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneClientException.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneClientException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneConstants.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneConstants.java index 16c37a5a..17790ca8 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneConstants.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneConstants.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; @@ -59,4 +59,7 @@ public interface KeystoneConstants { String AUTH_SERVICE_KEY = "service"; String AUTH_SERVICE_DEFAULT = "swift"; + + String AUTH_REGION_KEY = "region"; + String AUTH_REGION_DEFAULT = "regionOne"; } diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneInterruptedException.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneInterruptedException.java index d6bd6e2a..97ebabb0 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneInterruptedException.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneInterruptedException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneMapper.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneMapper.java index 1e4920af..c3127ca4 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneMapper.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneMapper.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneRequest.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneRequest.java index 9f4d6525..11cb7a50 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneRequest.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneRequest.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; @@ -28,11 +28,11 @@ * This class is specially structured in a way that is compatible with the * interface provided by the keystone service. Please refer to the keystone * documents for more detailed information.
- * + * * * {"auth": ... } * - * + * * @author qzheng */ public class KeystoneRequest { @@ -81,12 +81,12 @@ public void addTenantName(String tenantName) { * The authentication information holding either the password credentials or * the token id. This class is specially structured in a way that is * compatible with the interface provided by the keystone service.
- * + * * * {"passwordCredentials": ... , "token": ... , * "tenantId": "?", "tenantName": "?"} * - * + * * @author qzheng */ public static class AuthInfo { @@ -136,11 +136,11 @@ public void setTenantName(String tenantName) { * The credentials comprised of both the username and the password. This * class is specially structured in a way that is compatible with the * interface provided by the keystone service.
- * + * * - * {"username": "?", "password": "?"} + * {"username": "?", "password": "?"} * - * + * * @author qzheng */ public static class Credentials { @@ -174,11 +174,11 @@ public void setPassword(String password) { * The token meta data identified by its id. This class is specially * structured in a way that is compatible with the interface provided by * the keystone service.
- * + * * - * {"id": "?"} + * {"id": "?"} * - * + * * @author qzheng */ public static class Token { diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneResponse.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneResponse.java index ef284aa7..5faccdfb 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneResponse.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; @@ -192,6 +192,7 @@ public static class Endpoint { private String adminURL; private String internalURL; private String publicURL; + private String region; public String getAdminURL() { return adminURL; @@ -217,6 +218,16 @@ public void setPublicURL(String publicURL) { this.publicURL = publicURL; } + public String getRegion() { + return region; + } + + public void setRegion(String region) { + this.region = region; + } + + + } } diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneResponseException.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneResponseException.java index 34c31f73..70e111cb 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneResponseException.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneResponseException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneServerException.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneServerException.java index a829e0b9..3b6b3a41 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneServerException.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneServerException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneTimeoutException.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneTimeoutException.java index 4587dd2e..36c2c469 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneTimeoutException.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/KeystoneTimeoutException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/handler/AuthHandler.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/handler/AuthHandler.java index b93fa278..e1605bd9 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/handler/AuthHandler.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/handler/AuthHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone.handler; diff --git a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/handler/HttpAuthHandler.java b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/handler/HttpAuthHandler.java index b69b9490..3c8bf2b2 100644 --- a/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/handler/HttpAuthHandler.java +++ b/dev/cosbench-keystone/src/com/intel/cosbench/client/keystone/handler/HttpAuthHandler.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.keystone.handler; diff --git a/dev/cosbench-librados/META-INF/MANIFEST.MF b/dev/cosbench-librados/META-INF/MANIFEST.MF index d830a306..5e25fb0d 100644 --- a/dev/cosbench-librados/META-INF/MANIFEST.MF +++ b/dev/cosbench-librados/META-INF/MANIFEST.MF @@ -2,13 +2,13 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Librados Storage Client Bundle Bundle-SymbolicName: cosbench-librados -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Bundle-ClassPath: ., rados.jar, jna-3.5.2.jar -import-Package: com.intel.cosbench.api.auth, +Import-Package: com.intel.cosbench.api.auth, com.intel.cosbench.api.context, com.intel.cosbench.api.storage, com.intel.cosbench.client.http, diff --git a/dev/cosbench-librados/rados.jar b/dev/cosbench-librados/rados.jar index 51aaf257..b20cadff 100644 Binary files a/dev/cosbench-librados/rados.jar and b/dev/cosbench-librados/rados.jar differ diff --git a/dev/cosbench-librados/src/com/intel/cosbench/api/librados/LibradosStorage.java b/dev/cosbench-librados/src/com/intel/cosbench/api/librados/LibradosStorage.java index f2f1fcca..ab73fc93 100644 --- a/dev/cosbench-librados/src/com/intel/cosbench/api/librados/LibradosStorage.java +++ b/dev/cosbench-librados/src/com/intel/cosbench/api/librados/LibradosStorage.java @@ -1,5 +1,8 @@ -/** - +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -10,7 +13,22 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. + +*/ +/** + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ package com.intel.cosbench.api.librados; @@ -28,7 +46,9 @@ import com.ceph.rados.IoCTX; import com.ceph.rados.Rados; -import com.ceph.rados.RadosException; +import com.ceph.rados.exceptions.RadosAlreadyConnectedException; +import com.ceph.rados.exceptions.RadosException; +import com.ceph.rados.exceptions.RadosOperationInProgressException; import com.intel.cosbench.api.context.AuthContext; import com.intel.cosbench.api.storage.NoneStorage; import com.intel.cosbench.api.storage.StorageException; @@ -39,8 +59,8 @@ * LibradosStorage provides methods to access a Storage using librados. * It is based on rados-java * {@link https://github.com/wido/rados-java} - * - * + * + * * @author Niklas Goerke - niklas974@github * */ @@ -50,7 +70,7 @@ public class LibradosStorage extends NoneStorage { private String secretKey; private String endpoint; - private static Rados client; + private static Rados client; // as it's heavy to create a rados client, we will use shared rados client for all workers. public void init(Config config, Logger logger) { super.init(config, logger); @@ -64,17 +84,36 @@ public void init(Config config, Logger logger) { parms.put(AUTH_PASSWORD_KEY, secretKey); logger.debug("using storage config: {}", parms); - if (client == null) { - client = new Rados(this.accessKey); + try { + if (client == null) { + synchronized (this.getClass()) { + if(client == null) { + client = new Rados(this.accessKey); + client.confSet("key", this.secretKey); + client.confSet("mon_host", this.endpoint); + client.connect(); + logger.info("Librados client has connected."); + } + } + } + + logger.debug("Librados client has been initialized"); + } catch (RadosAlreadyConnectedException ace) { + logger.debug("The connection is already connected"); + } catch (RadosOperationInProgressException eip) { + logger.warn("Connection is in progress"); + // normally this means race condition where multiple threads are trying to use the same rados client to create connections. + // we will treat it's valid so far, but assume the later thread can directly use the connection after a short wait. try { - client.confSet("key", this.secretKey); - client.confSet("mon_host", this.endpoint); - client.connect(); - logger.debug("Librados client has been initialized"); - } catch (RadosException e) { - throw new StorageException(e); + Thread.sleep(100); + }catch(InterruptedException ie) { + throw new StorageException(ie); } + } catch (RadosException e) { + logger.error(e.getMessage()); + throw new StorageException(e); } + } public void setAuthContext(AuthContext info) { @@ -83,13 +122,12 @@ public void setAuthContext(AuthContext info) { public void dispose() { super.dispose(); -// client = null; } public InputStream getObject(String container, String object, Config config) { super.getObject(container, object, config); InputStream stream; - IoCTX ioctx; + IoCTX ioctx = null; try { ioctx = client.ioCtxCreate(container); long length = ioctx.stat(object).getSize(); @@ -97,14 +135,16 @@ public InputStream getObject(String container, String object, Config config) { throw new StorageException("Object larger than 2GB, handling not implemented"); // TODO: implement - read in parts and concatinate } - + byte[] buf = new byte[(int) length]; ioctx.read(object, (int) length, 0, buf); stream = new ByteArrayInputStream(buf); } catch (RadosException e) { throw new StorageException(e); + }finally { + if(ioctx != null) + client.ioCtxDestroy(ioctx); } - client.ioCtxDestroy(ioctx); return stream; } @@ -118,7 +158,7 @@ public void createContainer(String container, Config config) { } } if (!exists) { - client.poolCreate(container); + client.poolCreate(container); } } catch (RadosException e) { throw new StorageException(e); @@ -137,25 +177,32 @@ public void deleteContainer(String container, Config config) { public void createObject(String container, String object, InputStream data, long length, Config config) { super.createObject(container, object, data, length, config); byte[] buf = new byte[(int) length]; + IoCTX ioctx = null; try { data.read(buf, 0, (int) length); - IoCTX ioctx = client.ioCtxCreate(container); + ioctx = client.ioCtxCreate(container); ioctx.write(object, buf); } catch (RadosException e) { throw new StorageException(e); } catch (IOException e) { throw new StorageException(e); + } finally { + if(ioctx != null) + client.ioCtxDestroy(ioctx); } } public void deleteObject(String container, String object, Config config) { super.deleteObject(container, object, config); - IoCTX ioctx; + IoCTX ioctx = null; try { ioctx = client.ioCtxCreate(container); ioctx.remove(object); } catch (RadosException e) { throw new StorageException(e); + } finally { + if(ioctx != null) + client.ioCtxDestroy(ioctx); } } } diff --git a/dev/cosbench-librados/src/com/intel/cosbench/api/librados/LibradosStorageFactory.java b/dev/cosbench-librados/src/com/intel/cosbench/api/librados/LibradosStorageFactory.java index d90756dd..d2167c00 100644 --- a/dev/cosbench-librados/src/com/intel/cosbench/api/librados/LibradosStorageFactory.java +++ b/dev/cosbench-librados/src/com/intel/cosbench/api/librados/LibradosStorageFactory.java @@ -1,5 +1,8 @@ -/** - +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -10,14 +13,29 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. + +*/ +/** + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ package com.intel.cosbench.api.librados; import com.intel.cosbench.api.storage.*; /** - * + * * @author Niklas Goerke - niklas974@github * */ diff --git a/dev/cosbench-librados/src/com/intel/cosbench/client/librados/LibradosConstants.java b/dev/cosbench-librados/src/com/intel/cosbench/client/librados/LibradosConstants.java index 3ca9a49f..ce2d41b0 100644 --- a/dev/cosbench-librados/src/com/intel/cosbench/client/librados/LibradosConstants.java +++ b/dev/cosbench-librados/src/com/intel/cosbench/client/librados/LibradosConstants.java @@ -1,4 +1,22 @@ -/** +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -9,12 +27,12 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.client.librados; /** - * + * * @author Niklas Goerke - niklas974@github * */ diff --git a/dev/cosbench-librados/test/com/intel/cosbench/api/librados/DummyConfig.java b/dev/cosbench-librados/test/com/intel/cosbench/api/librados/DummyConfig.java index cb6ebf12..cb019f4d 100644 --- a/dev/cosbench-librados/test/com/intel/cosbench/api/librados/DummyConfig.java +++ b/dev/cosbench-librados/test/com/intel/cosbench/api/librados/DummyConfig.java @@ -1,4 +1,8 @@ -/** +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -9,8 +13,22 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. + +*/ +/** +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ package com.intel.cosbench.api.librados; import static com.intel.cosbench.client.librados.LibradosConstants.AUTH_PASSWORD_KEY; @@ -19,16 +37,16 @@ import com.intel.cosbench.config.Config; /** - * + * * @author Niklas Goerke niklas974@github - * + * */ public class DummyConfig implements Config { private final String endpoint = "127.0.0.1"; private final String accessKey = "admin"; private final String secretKey = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="; - + @Override public String get(String key) { return null; @@ -46,7 +64,7 @@ public String get(String key, String value) { } else { return null; } - + } @Override diff --git a/dev/cosbench-librados/test/com/intel/cosbench/api/librados/DummyLogger.java b/dev/cosbench-librados/test/com/intel/cosbench/api/librados/DummyLogger.java index f043c371..f3264597 100644 --- a/dev/cosbench-librados/test/com/intel/cosbench/api/librados/DummyLogger.java +++ b/dev/cosbench-librados/test/com/intel/cosbench/api/librados/DummyLogger.java @@ -1,4 +1,22 @@ -/** +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -9,15 +27,15 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.librados; import com.intel.cosbench.log.Logger; /** - * + * * @author Niklas Goerke niklas974@github - * + * */ public class DummyLogger implements Logger { diff --git a/dev/cosbench-librados/test/com/intel/cosbench/api/librados/TestLibradosStorage.java b/dev/cosbench-librados/test/com/intel/cosbench/api/librados/TestLibradosStorage.java index 4ecbe863..2be0794b 100644 --- a/dev/cosbench-librados/test/com/intel/cosbench/api/librados/TestLibradosStorage.java +++ b/dev/cosbench-librados/test/com/intel/cosbench/api/librados/TestLibradosStorage.java @@ -1,4 +1,22 @@ -/** +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -9,7 +27,7 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. +limitations under the License. */ package com.intel.cosbench.api.librados; @@ -26,13 +44,13 @@ import org.junit.Before; import org.junit.Test; -import com.ceph.rados.RadosException; +import com.ceph.rados.exceptions.RadosException; import com.intel.cosbench.api.storage.StorageException; /** - * + * * @author Niklas Goerke niklas974@github - * + * */ public class TestLibradosStorage { diff --git a/dev/cosbench-log/META-INF/MANIFEST.MF b/dev/cosbench-log/META-INF/MANIFEST.MF index ca2512c1..fc26e9f7 100644 --- a/dev/cosbench-log/META-INF/MANIFEST.MF +++ b/dev/cosbench-log/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Log Bundle Bundle-SymbolicName: cosbench-log -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Activator: com.intel.cosbench.log.osgi.Activator Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 diff --git a/dev/cosbench-log/src/com/intel/cosbench/log/LogFactory.java b/dev/cosbench-log/src/com/intel/cosbench/log/LogFactory.java index 37b30f13..ec9b01df 100644 --- a/dev/cosbench-log/src/com/intel/cosbench/log/LogFactory.java +++ b/dev/cosbench-log/src/com/intel/cosbench/log/LogFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,17 +12,17 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.log; /** * A Factory class to create different LogManager based on log name, by default * log4j LogManager will use. - * + * * @author ywang19, qzheng7 - * + * */ public class LogFactory { @@ -36,7 +36,7 @@ public static LogManager createLogManager() { /** * The method creates one LogManager instance by provided class name. - * + * * @param logmgr * the class name for log manager * @return one new LogManager instance created by logmgr name, if any diff --git a/dev/cosbench-log/src/com/intel/cosbench/log/LogLevel.java b/dev/cosbench-log/src/com/intel/cosbench/log/LogLevel.java index ffcef3e7..7a912ecd 100644 --- a/dev/cosbench-log/src/com/intel/cosbench/log/LogLevel.java +++ b/dev/cosbench-log/src/com/intel/cosbench/log/LogLevel.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.log; /** * The enumeration of different log levels, five levels are defined: TRACE, DEBUG, INFO, WARN, ERROR. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-log/src/com/intel/cosbench/log/LogManager.java b/dev/cosbench-log/src/com/intel/cosbench/log/LogManager.java index 14b89a6e..032e5940 100644 --- a/dev/cosbench-log/src/com/intel/cosbench/log/LogManager.java +++ b/dev/cosbench-log/src/com/intel/cosbench/log/LogManager.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.log; @@ -21,9 +21,9 @@ /** * The interface of LogManager. - * + * * @author ywang19, qzheng7 - * + * */ public interface LogManager { diff --git a/dev/cosbench-log/src/com/intel/cosbench/log/Logger.java b/dev/cosbench-log/src/com/intel/cosbench/log/Logger.java index a9a601c2..56e80476 100644 --- a/dev/cosbench-log/src/com/intel/cosbench/log/Logger.java +++ b/dev/cosbench-log/src/com/intel/cosbench/log/Logger.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,14 +12,14 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.log; /** * The interface of Logger. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jLogManager.java b/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jLogManager.java index b4812b06..e1cb0996 100644 --- a/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jLogManager.java +++ b/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jLogManager.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.log.log4j; @@ -32,9 +32,9 @@ * The wrapper of log4j LogManager; this log manager will create log file with * maximum size of 10 MB, and the maximum number of backup file is 10. The * default log level is INFO. - * + * * @author ywang19, qzheng7 - * + * */ public class Log4jLogManager implements LogManager { diff --git a/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jLogger.java b/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jLogger.java index db675b66..a788ee7a 100644 --- a/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jLogger.java +++ b/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jLogger.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.log.log4j; @@ -24,7 +24,7 @@ /** * The wrapper of log4j logger. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jUtils.java b/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jUtils.java index 6aac806b..3cb1cd56 100644 --- a/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jUtils.java +++ b/dev/cosbench-log/src/com/intel/cosbench/log/log4j/Log4jUtils.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.log.log4j; @@ -25,7 +25,7 @@ public class Log4jUtils { /** * Converts cosbench log level to log4j log level. - * + * * @param level * cosbench log level * @return corresponding log4j log level. @@ -47,7 +47,7 @@ public static Level toLog4jLevel(LogLevel level) { /** * Converts log4j log level to cosbench log level - * + * * @param level * log4j log level * @return corresponding cosbench log level diff --git a/dev/cosbench-log/src/com/intel/cosbench/log/osgi/Activator.java b/dev/cosbench-log/src/com/intel/cosbench/log/osgi/Activator.java index b3b076ac..dd55ad8b 100644 --- a/dev/cosbench-log/src/com/intel/cosbench/log/osgi/Activator.java +++ b/dev/cosbench-log/src/com/intel/cosbench/log/osgi/Activator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.log.osgi; @@ -25,7 +25,7 @@ /** * The bundle activator for cosbench log. * cosbench log bundle includes a few interfaces, and one log4j based implementation. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-log4j/META-INF/MANIFEST.MF b/dev/cosbench-log4j/META-INF/MANIFEST.MF index d889c885..7d300cd7 100644 --- a/dev/cosbench-log4j/META-INF/MANIFEST.MF +++ b/dev/cosbench-log4j/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Log4j Configuration Bundle Bundle-SymbolicName: cosbench-log4j -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Fragment-Host: com.springsource.org.apache.log4j;bundle-version="[1.0.0,2.0.0)" Bundle-RequiredExecutionEnvironment: JavaSE-1.6 diff --git a/dev/cosbench-manta/.gitignore b/dev/cosbench-manta/.gitignore new file mode 100644 index 00000000..7cba7e59 --- /dev/null +++ b/dev/cosbench-manta/.gitignore @@ -0,0 +1,128 @@ +# Created by https://www.gitignore.io/api/intellij,eclipse,java,maven + +### Intellij ### +# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm + +*.iml + +## Directory-based project format: +.idea/ +# if you remove the above rule, at least ignore the following: + +# User-specific stuff: +# .idea/workspace.xml +# .idea/tasks.xml +# .idea/dictionaries +# .idea/shelf + +# Sensitive or high-churn files: +# .idea/dataSources.ids +# .idea/dataSources.xml +# .idea/sqlDataSources.xml +# .idea/dynamic.xml +# .idea/uiDesigner.xml + +# Gradle: +# .idea/gradle.xml +# .idea/libraries + +# Mongo Explorer plugin: +# .idea/mongoSettings.xml + +## File-based project format: +*.ipr +*.iws + +## Plugin-specific files: + +# IntelliJ +/out/ + +# mpeltonen/sbt-idea plugin +.idea_modules/ + +# JIRA plugin +atlassian-ide-plugin.xml + +# Crashlytics plugin (for Android Studio and IntelliJ) +com_crashlytics_export_strings.xml +crashlytics.properties +crashlytics-build.properties +fabric.properties + + +### Eclipse ### +*.pydevproject +.metadata +.gradle +bin/ +tmp/ +*.tmp +*.bak +*.swp +*~.nib +local.properties +.settings/ +.loadpath + +# Eclipse Core +.project + +# External tool builders +.externalToolBuilders/ + +# Locally stored "Eclipse launch configurations" +*.launch + +# CDT-specific +.cproject + +# JDT-specific (Eclipse Java Development Tools) +.classpath + +# Java annotation processor (APT) +.factorypath + +# PDT-specific +.buildpath + +# sbteclipse plugin +.target + +# TeXlipse plugin +.texlipse + +# STS (Spring Tool Suite) +.springBeans + + +### Java ### +*.class + +# Mobile Tools for Java (J2ME) +.mtj.tmp/ + +# Package Files # +*.jar +*.war +*.ear + +# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml +hs_err_pid* + + +### Maven ### +target/ +pom.xml.tag +pom.xml.releaseBackup +pom.xml.versionsBackup +pom.xml.next +release.properties +dependency-reduced-pom.xml +buildNumber.properties +.mvn/timing.properties + +# Allow everything in the Docker build bin directory +!docker_build/usr/local/bin +# Allow all of the local dependencies +!build-dependencies/** diff --git a/dev/cosbench-manta/.travis.yml b/dev/cosbench-manta/.travis.yml new file mode 100644 index 00000000..48efff0c --- /dev/null +++ b/dev/cosbench-manta/.travis.yml @@ -0,0 +1,21 @@ +language: java +sudo: false + +before_install: + - echo 'MAVEN_OPTS="-Dorg.slf4j.simpleLogger.defaultLogLevel=warn"' >~/.mavenrc + +matrix: + fast_finish: true + include: + # checkstyle + - jdk: oraclejdk8 + env: + - DESC="checkstyle" + - CMD="mvn clean checkstyle:check" + # unit tests (oraclejdk8) + - jdk: oraclejdk8 + env: + - DESC="oraclejdk8 package" + - CMD="mvn clean package -Dcheckstyle.skip=true" + +script: echo ${CMD}; ${CMD} diff --git a/dev/cosbench-manta/CHANGELOG.md b/dev/cosbench-manta/CHANGELOG.md new file mode 100644 index 00000000..5819523f --- /dev/null +++ b/dev/cosbench-manta/CHANGELOG.md @@ -0,0 +1,13 @@ +# Change Log +All notable changes to this project will be documented in this file. +This project adheres to [Semantic Versioning](http://semver.org/). + +## [1.0.1] - 2015-12-22 +### Fixed +- Fixed OSGI import package scoping issue + +## [1.0.0] - 2015-12-22 +### Added +- Initial check in of benchmarking functionality. +- Added Dockerfile support. +- Added sane build system using Maven. diff --git a/dev/cosbench-manta/Dockerfile b/dev/cosbench-manta/Dockerfile new file mode 100644 index 00000000..e71b5d91 --- /dev/null +++ b/dev/cosbench-manta/Dockerfile @@ -0,0 +1,71 @@ +# dekobon/cosbench-manta + +# We use an Ubuntu OS because it is the reference OS in the COSBench documentation. +# We use the phusion passenger base image of Ubuntu, so that we can run COSBench +# as a multi-process container. +FROM phusion/baseimage:0.9.18 + +MAINTAINER Elijah Zupancic + +ENV JAVA_MAJOR_VERSION 8 +ENV COSBENCH_VERSION 0.4.1.0 +ENV COSBENCH_CHECKSUM a044cd232b3cc376802aa6a4a697988ec690a8b1d70040641710066acd322c5a +ENV COSBENCH_MANTA_VERSION 1.0.1 +ENV COSBENCH_MANTA_CHECKSUM 01aee2bbf02fe95c1bb8f1d6dc88aff6ef4b29f16836d4a5dc06d29c2fe41805 + +# Setup the (Oracle) JVM and install needed utilities +RUN echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections +RUN echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections + +RUN apt-add-repository ppa:webupd8team/java && \ + apt-get -qq update && \ + apt-get -qy upgrade && \ + apt-get install -y oracle-java${JAVA_MAJOR_VERSION}-installer patch unzip dc && \ + apt-get autoremove -y && \ + apt-get clean && \ + rm -rf /var/lib/apt/lists/* \ + /tmp/* \ + /var/tmp/* \ + /var/cache/oracle-jdk* \ + /usr/lib/jvm/java-${JAVA_MAJOR_VERSION}-oracle/src.zip \ + /usr/lib/jvm/java-${JAVA_MAJOR_VERSION}-oracle/javafx-src.zip + +# Download and install Cosbench +RUN curl --retry 6 -Ls "https://github.com/intel-cloud/cosbench/releases/download/v${COSBENCH_VERSION}/${COSBENCH_VERSION}.zip" > /tmp/cosbench.zip && \ + echo "${COSBENCH_CHECKSUM} /tmp/cosbench.zip" | sha256sum -c && \ + unzip -q /tmp/cosbench.zip -d /opt/ && \ + mv "/opt/${COSBENCH_VERSION}" /opt/cosbench && \ + rm /tmp/cosbench.zip + +# Download and install the Manta adaptor +RUN curl --retry 6 -Ls "https://github.com/joyent/cosbench-manta/releases/download/cosbench-manta-${COSBENCH_MANTA_VERSION}/cosbench-manta-${COSBENCH_MANTA_VERSION}.jar" > /opt/cosbench/osgi/plugins/cosbench-manta.jar && \ + echo "${COSBENCH_MANTA_CHECKSUM} /opt/cosbench/osgi/plugins/cosbench-manta.jar" | sha256sum -c + +# Adding machine sizing utility useful when on Triton +COPY docker_build/usr/local/bin /usr/local/bin +RUN chmod +x /usr/local/bin/proclimit + +# Install patches that will update configuration for use with Manta +# benchmarking. Check this directory for what we had to change to enable +# the adapter. +COPY docker_build/patches /patches + +# Adding sample Manta configuration and init files +COPY docker_build/opt/cosbench /opt/cosbench + +# Patch Cosbench for use with the Manta adaptor +RUN patch -p0 < /patches/manta_enabled.patch + +# Setup Tomcat user to run COSBench process +RUN groupadd -g 120 tomcat && \ + useradd -g 120 -u 120 -c 'Tomcat User' -d /opt/cosbench -r -s /bin/false tomcat && \ + mkdir /opt/cosbench/.ssh && \ + chown -R tomcat:tomcat /opt/cosbench + +# Run the container using the tomcat user by default +USER tomcat + +# COSBench driver port +EXPOSE 18088 +# COSBench controller port +EXPOSE 19088 diff --git a/dev/cosbench-manta/LICENSE b/dev/cosbench-manta/LICENSE new file mode 100644 index 00000000..11ca4aa7 --- /dev/null +++ b/dev/cosbench-manta/LICENSE @@ -0,0 +1,20 @@ +The MIT License (MIT) +Copyright (c) 2015 Joyent + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of +the Software, and to permit persons to whom the Software is furnished to do so, +subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/dev/cosbench-manta/README.md b/dev/cosbench-manta/README.md new file mode 100644 index 00000000..34ef58d2 --- /dev/null +++ b/dev/cosbench-manta/README.md @@ -0,0 +1,228 @@ +[![Build Status](https://travis-ci.org/joyent/cosbench-manta.svg?branch=master)](https://travis-ci.org/joyent/cosbench-manta) + +# COSBench Manta Adaptor + +This project provides an [adaptor](https://github.com/intel-cloud/cosbench/blob/master/COSBench-Adaptor-Dev-Guide.odt) +for benchmarking [Joyent's object store](https://www.joyent.com/object-storage) [Manta](https://github.com/joyent/manta) +using [COSBench](https://github.com/intel-cloud/cosbench/). + +## Requirements +* [COSBench 0.4.1.0](https://github.com/intel-cloud/cosbench/releases/tag/v0.4.1.0) +* [Java 1.8](http://www.oracle.com/technetwork/java/javase/downloads/index.html) or higher. +* [Maven 3.3.x](https://maven.apache.org/) + +## Building from Source +If you prefer to build from source, you'll also need +[Maven](https://maven.apache.org/), and then invoke: + +``` bash +# mvn package +``` + +This should generate an OSGI bundle inside the `./target` directory that you +can drop into the COSBench `osgi/plugins`` directory. + +## Configuration + +Sample COSBench job configuration files are available in the `./docker_buikd/opt/cosbench/conf` directory. + +## Docker +You can use a preconfigured host with COSBench and the Manta adaptor preinstalled +when you run the project's [Docker](https://www.docker.com/) image: +[`dekobon/cosbench-manta`](https://hub.docker.com/r/dekobon/cosbench-manta/). + +Using Docker on your local machine you can log into a COSBench system setup for +Manta by doing: + +``` +# Adjust the key paths if needed and be sure to specify your own MANTA_USER +docker run --name=cosbench \ + -e "MANTA_PUBLIC_KEY=$(cat $HOME/.ssh/id_rsa.pub)" \ + -e "MANTA_PRIVATE_KEY=$(cat $HOME/.ssh/id_rsa)" \ + -e "MANTA_URL=https://us-east.manta.joyent.com:443" \ + -e MANTA_USER=username \ + -p 18088:18088 -p 19088:19088 \ + -it dekobon/cosbench-manta:latest bash +``` + +Using Triton on the Joyent public cloud you can log into a COSBench system setup for +Manta by doing: +``` +# Adjust the key paths if needed and be sure to specify your own MANTA_USER +# This will create a container with a 8gb memory package size and size the +# JVM to fit within that range +docker run --name=cosbench \ + -e "MANTA_PUBLIC_KEY=$(cat $HOME/.ssh/id_rsa.pub)" \ + -e "MANTA_PRIVATE_KEY=$(cat $HOME/.ssh/id_rsa)" \ + -e "MANTA_URL=https://us-east.manta.joyent.com:443" \ + -e MANTA_USER=username \ + -e "JAVA_OPTS=-Xmx3800m" \ + -m 8g \ + -p 18088:18088 -p 19088:19088 \ + -it dekobon/cosbench-manta:latest bash +``` + +Note: you will **need** to specify the Manta environment variables in order for +the Manta adaptor to work correctly. You can override or add to the configuration +within the workload configuration, but you can't embed public nor private keys +in that configuration, so you will ultimately have to set some of these environment +variables. + +Using this configuration you will be able to run the COSBench startup script by: +``` +cd /opt/cosbench +sh ./start-all.sh + +# If I am on Triton, it would be useful to note the public IP address +ifconfig + +# When I want to shut down: +sh ./stop-all.sh +``` + +Then you will be able to go to the driver control panel ([http://server:18088/driver/index.html](http://localhost:18088/driver/index.html)) or the +more useful controller control panel ([http://server:19088/controller/](http://localhost:19088/controller/)). +From the controller control panel, you can run a benchmark by submitting a new workload. +The workload configuration `opt/cosbench/conf/manta-config.xml` is a good +example to get started with. + +## Contributions + +Contributions welcome! Please ensure that `# mvn checkstyle:checkstyle -Dcheckstyle.skip=false` runs +clean with no warnings or errors. + +## Testing + +There are no unit tests for this project yet. This would be a great first contribution! + +## Releasing the Java Components + +In order to release to [Maven central](https://search.maven.org/), you will need [an account] (https://issues.sonatype.org) with [Sonatype OSSRH](http://central.sonatype.org/pages/ossrh-guide.html). +If you do not already have an account, you can click the signup link from the login screen +to begin the process of registering for an account. After signing up, you will need to add +your sonatype credentials to your your maven settings file. By default this settings file is +located at `$HOME/.m2/settings.xml`. In addition to sonatype credentials, you will +also need to add a [gpg signing](https://maven.apache.org/plugins/maven-gpg-plugin/sign-mojo.html) key configuration. + +For the security conscious, a [guide to encrypting credentials in maven settings files](https://maven.apache.org/guides/mini/guide-encryption.html) exists to +illustrate how credentials can be protected. + +The following is an example settings.xml file: + +```xml + + + + gpg + + + gpg + keyname + passphrase + ${env.HOME}/.gnupg/secring.gpg + + + + + + ossrh + username + password + + + +``` + +To perform a release: + +1. Make sure the source builds, test suites pass, and the source and java artifacts can + be generated and signed: +`mvn clean verify -Prelease` +2. Start from a clean working directory and make sure you have no modified +files in your workspace: +`mvn clean && git status` +3. Prepare the release: +4. `mvn release:clean release:prepare` +4. Enter the version to be associated with this release. +You should be prompted for this version number, and the default assumed version +will be shown and should correspond to the version that was in the pom.xml +file but WITHOUT the `-SNAPSHOT` suffix. +5. Enter the SCM tag to be used to mark this commit in the SCM. +You should be prompted for this tag name, and the default will be +`{projectName}-{releaseVersion}` +6. Enter the new development version. +You should be prompted for this version number, and the default for this will +be an incremented version number of the release followed by a `-SNAPSHOT` +suffix. + + At this point + * The release plugin will continue and build/package the artifacts. + * The pom.xml file will be updated to reflect the version change to the release +version. + * The new pom.xml will be committed. + * The new commit will be tagged. + * The pom.xml file will be updated again with the new development version. + * Then this new pom.xml will be committed. + + If the process fails for some reason during any of these points, you can invoke +`mvn release:rollback` to go back to the preparation point and try again, but +you will also have to revert any SCM commits that were done +(`git reset --hard HEAD^1` command works well for this) as well as remove any +tags that were created (`git tag -l && git tag -d ` commands help +with this). + +7. Push tags to github: +`git push --follow-tags` +In order for the `release:perform` goal to complete successfully, you will need to +push the tags created by the maven release plugin to the remote git server. + +8. Perform the actual release: +`mvn release:perform` +A build will be performed and packaged and artifacts deployed to the sonatype +staging repository. + +9. Log into the [Sonatype OSSHR Next](https://oss.sonatype.org) web interface +to [verify and promote](http://central.sonatype.org/pages/releasing-the-deployment.html) +the build. + +**NOTE**: By default, these instructions assumes the release is being done from a +branch that can be merged into a primary branch upon successful completion, +and that the SCM operations that are carried out by maven plugins will NOT +access the repo, but rather, work on a local copy instead. The release plugin +as configured in the maven repo sets values for this assumption +(`localCheckout=true` and `pushChanges=false`). + +**NOTE**: If the release is being done in a separate fork of the primary +github repo, doing a merge via pull request will not also copy the tags that +were created during the release process. The tags will have to be created in +the primary repo separately, but this may be preferred anyway. + +### Bugs + +See . + +## License + +The MIT License (MIT) +Copyright (c) 2015 Joyent + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of +the Software, and to permit persons to whom the Software is furnished to do so, +subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar new file mode 100644 index 00000000..641c35b2 Binary files /dev/null and b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar differ diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar.md5 new file mode 100644 index 00000000..51030394 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar.md5 @@ -0,0 +1 @@ +aab23f2530b736717c827276809c42b6 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar.sha1 new file mode 100644 index 00000000..9b41be3e --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.jar.sha1 @@ -0,0 +1 @@ +c4b1d58b2ba3ecd12f709dea549cc47db47d2425 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom new file mode 100644 index 00000000..877876aa --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom @@ -0,0 +1,8 @@ + + + 4.0.0 + com.intel.cosbench + cosbench-api + 0.4.1.0 + diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom.md5 new file mode 100644 index 00000000..03b40778 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom.md5 @@ -0,0 +1 @@ +b92adee9f789e2e89b383573ec7f7ae0 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom.sha1 new file mode 100644 index 00000000..4c07e284 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/0.4.1.0/cosbench-api-0.4.1.0.pom.sha1 @@ -0,0 +1 @@ +adf9dfc2d621920529186335ede0390582dcf3e0 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml new file mode 100644 index 00000000..d97d754f --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml @@ -0,0 +1,12 @@ + + + com.intel.cosbench + cosbench-api + + 0.4.1.0 + + 0.4.1.0 + + 20151222211546 + + diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml.md5 new file mode 100644 index 00000000..dfb8d8c9 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml.md5 @@ -0,0 +1 @@ +b3c79a6685deb254ef8d9bc1133a14fd \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml.sha1 new file mode 100644 index 00000000..08f0c0d8 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-api/maven-metadata.xml.sha1 @@ -0,0 +1 @@ +576a125813bbfb2d9cb55872e114979c4761d111 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar new file mode 100644 index 00000000..bac38813 Binary files /dev/null and b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar differ diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar.md5 new file mode 100644 index 00000000..d33f54a0 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar.md5 @@ -0,0 +1 @@ +d15c36ab947f01f044a2800e6fefb7e0 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar.sha1 new file mode 100644 index 00000000..10ba01dd --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.jar.sha1 @@ -0,0 +1 @@ +733ca91b123ce73dac9948178627283fc0856ba9 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom new file mode 100644 index 00000000..958a3817 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom @@ -0,0 +1,8 @@ + + + 4.0.0 + com.intel.cosbench + cosbench-config + 0.4.1.0 + diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom.md5 new file mode 100644 index 00000000..2afb9e02 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom.md5 @@ -0,0 +1 @@ +1229c360b319e0284b5209e0ac69e163 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom.sha1 new file mode 100644 index 00000000..4e029e1d --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/0.4.1.0/cosbench-config-0.4.1.0.pom.sha1 @@ -0,0 +1 @@ +5bd61b965d56025db2cab32e0e14030f375d62eb \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml new file mode 100644 index 00000000..421d0bc8 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml @@ -0,0 +1,12 @@ + + + com.intel.cosbench + cosbench-config + + 0.4.1.0 + + 0.4.1.0 + + 20151222211652 + + diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml.md5 new file mode 100644 index 00000000..4c7b4bb1 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml.md5 @@ -0,0 +1 @@ +f569e2490f551579d34b0ffc69692c88 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml.sha1 new file mode 100644 index 00000000..235e0b8a --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-config/maven-metadata.xml.sha1 @@ -0,0 +1 @@ +bc40a0e2f729fec121ec8da440da4941ee4f6080 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar new file mode 100644 index 00000000..80436599 Binary files /dev/null and b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar differ diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar.md5 new file mode 100644 index 00000000..c0557497 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar.md5 @@ -0,0 +1 @@ +a5e503a7fdc04a2e47789e664edcd934 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar.sha1 new file mode 100644 index 00000000..37cda199 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.jar.sha1 @@ -0,0 +1 @@ +0c1acd2375b0b6496a8e11d3c7aa49656f90494d \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom new file mode 100644 index 00000000..b5fae869 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom @@ -0,0 +1,8 @@ + + + 4.0.0 + com.intel.cosbench + cosbench-log + 0.4.1.0 + diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom.md5 new file mode 100644 index 00000000..bdc7cef5 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom.md5 @@ -0,0 +1 @@ +c710fb47d76e81c884d321289f9100fc \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom.sha1 new file mode 100644 index 00000000..cf1be2d9 --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/0.4.1.0/cosbench-log-0.4.1.0.pom.sha1 @@ -0,0 +1 @@ +5c6155a24b47c93212555ebe20a6538b0f75b186 \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml new file mode 100644 index 00000000..ae829c8f --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml @@ -0,0 +1,12 @@ + + + com.intel.cosbench + cosbench-log + + 0.4.1.0 + + 0.4.1.0 + + 20151222211911 + + diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml.md5 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml.md5 new file mode 100644 index 00000000..327e4c4b --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml.md5 @@ -0,0 +1 @@ +efb0908dad3500101b22edcce97360af \ No newline at end of file diff --git a/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml.sha1 b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml.sha1 new file mode 100644 index 00000000..5760363f --- /dev/null +++ b/dev/cosbench-manta/build-dependencies/com/intel/cosbench/cosbench-log/maven-metadata.xml.sha1 @@ -0,0 +1 @@ +fd5989f8c4cc7ca9031f65f64877ba8bead9ee0c \ No newline at end of file diff --git a/dev/cosbench-manta/build.properties b/dev/cosbench-manta/build.properties new file mode 100644 index 00000000..fc457829 --- /dev/null +++ b/dev/cosbench-manta/build.properties @@ -0,0 +1,9 @@ +source.. = src/ +output.. = bin/ +bin.includes = META-INF/,\ + .,\ + bcpkix-jdk15on-1.53.jar,\ + bcprov-jdk15on-1.53.jar,\ + java-manta-client-2.0.0.jar,\ + slf4j-api-1.7.7.jar,\ + slf4j-log4j12-1.7.7.jar diff --git a/dev/cosbench-manta/checkstyle.xml b/dev/cosbench-manta/checkstyle.xml new file mode 100644 index 00000000..9428dce8 --- /dev/null +++ b/dev/cosbench-manta/checkstyle.xml @@ -0,0 +1,176 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/cosbench-manta/docker_build/opt/cosbench/.bashrc b/dev/cosbench-manta/docker_build/opt/cosbench/.bashrc new file mode 100644 index 00000000..1136bbee --- /dev/null +++ b/dev/cosbench-manta/docker_build/opt/cosbench/.bashrc @@ -0,0 +1,50 @@ +MANTA_ENV_WORKING=1 + +if [ -n "${MANTA_PUBLIC_KEY}" ] && [ ! -f $HOME/.ssh/id_rsa.pub ]; then + echo "${MANTA_PUBLIC_KEY}" > $HOME/.ssh/id_rsa.pub + chmod og-rwx $HOME/.ssh/id_rsa.pub + unset MANTA_PUBLIC_KEY +fi + +if [ -n "${MANTA_PRIVATE_KEY}" ] && [ ! -f $HOME/.ssh/id_rsa ]; then + echo "${MANTA_PRIVATE_KEY}" > $HOME/.ssh/id_rsa + chmod og-rwx $HOME/.ssh/id_rsa + unset MANTA_PRIVATE_KEY +fi + +if [ ! -f $HOME/.ssh/id_rsa.pub ]; then + >&2 echo "No public key for Manta authentication set at $HOME/.ssh/id_rsa.pub" + MANTA_ENV_WORKING=0 +fi + +if [ -z "${MANTA_USER}" ]; then + >&2 echo "MANTA_USER must be set in order for Manta adaptor to work" + MANTA_ENV_WORKING=0 +fi + +if [ -z "${MANTA_URL}" ]; then + >&2 echo "MANTA_URL is not set. Defaulting to: https://us-east.manta.joyent.com:443" + export MANTA_URL=https://us-east.manta.joyent.com:443 +fi + +if [ $MANTA_ENV_WORKING -ne 1 ]; then + >&2 echo "Manta environment is not setup correctly" +fi + +export MANTA_KEY_ID=$(ssh-keygen -l -f $HOME/.ssh/id_rsa.pub | awk '{print $2}') + +# If we are running on Triton, then we will tune the JVM for the platform +if [ -d /native ]; then + HW_THREADS=$(/usr/local/bin/proclimit) + + if [ $HW_THREADS -le 8 ]; then + GC_THREADS=$(echo "8k $HW_THREADS 2 + pq" | dc) + else + # ParallelGCThreads = (ncpus <= 8) ? ncpus : 3 + ((ncpus * 5) / 8) + ADJUSTED=$(echo "8k $HW_THREADS 5 * pq" | dc) + DIVIDED=$(echo "8k $ADJUSTED 8 / pq" | dc) + GC_THREADS=$(echo "8k $DIVIDED 3 + pq" | dc | awk 'function ceil(valor) { return (valor == int(valor) && value != 0) ? valor : int(valor)+1 } { printf "%d", ceil($1) }') + fi + + export JAVA_OPTS="$JAVA_OPTS -XX:-UseGCTaskAffinity -XX:-BindGCTaskThreadsToCPUs -XX:ParallelGCThreads=${GC_THREADS}" +fi diff --git a/dev/cosbench-manta/docker_build/opt/cosbench/conf/manta-config.xml b/dev/cosbench-manta/docker_build/opt/cosbench/conf/manta-config.xml new file mode 100644 index 00000000..640e354d --- /dev/null +++ b/dev/cosbench-manta/docker_build/opt/cosbench/conf/manta-config.xml @@ -0,0 +1,33 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/cosbench-manta/docker_build/patches/manta_enabled.patch b/dev/cosbench-manta/docker_build/patches/manta_enabled.patch new file mode 100644 index 00000000..ceacdfd9 --- /dev/null +++ b/dev/cosbench-manta/docker_build/patches/manta_enabled.patch @@ -0,0 +1,56 @@ ++++ /opt/cosbench/conf/.driver/config.ini +@@ -78,7 +78,8 @@ + plugins/cosbench-core@8\:start,\ + plugins/cosbench-driver@8\:start,\ + plugins/cosbench-core-web@8\:start,\ +-plugins/cosbench-driver-web@8\:start ++plugins/cosbench-driver-web@8\:start,\ ++plugins/cosbench-manta@7\:start + osgi.clean=true + osgi.noShutdown=true + osgi.startLevel=8 ++++ /opt/cosbench/conf/manta-config.xml +@@ -0,0 +1,33 @@ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++ ++++ /opt/cosbench/start-driver.sh +@@ -22,7 +22,7 @@ + SERVICE_NAME=driver + VERSION=`cat VERSION` + +-OSGI_BUNDLES="cosbench-log_${VERSION} cosbench-tomcat_${VERSION} cosbench-config_${VERSION} cosbench-http_${VERSION} cosbench-cdmi-util_${VERSION} cosbench-core_${VERSION} cosbench-core-web_${VERSION} cosbench-api_${VERSION} cosbench-mock_${VERSION} cosbench-ampli_${VERSION} cosbench-swift_${VERSION} cosbench-keystone_${VERSION} cosbench-httpauth_${VERSION} cosbench-s3_${VERSION} cosbench-librados_${VERSION} cosbench-scality_${VERSION} cosbench-cdmi-swift_${VERSION} cosbench-cdmi-base_${VERSION} cosbench-driver_${VERSION} cosbench-driver-web_${VERSION}" ++OSGI_BUNDLES="cosbench-log_${VERSION} cosbench-tomcat_${VERSION} cosbench-config_${VERSION} cosbench-http_${VERSION} cosbench-cdmi-util_${VERSION} cosbench-core_${VERSION} cosbench-core-web_${VERSION} cosbench-api_${VERSION} cosbench-mock_${VERSION} cosbench-ampli_${VERSION} cosbench-swift_${VERSION} cosbench-keystone_${VERSION} cosbench-httpauth_${VERSION} cosbench-s3_${VERSION} cosbench-manta cosbench-librados_${VERSION} cosbench-scality_${VERSION} cosbench-cdmi-swift_${VERSION} cosbench-cdmi-base_${VERSION} cosbench-driver_${VERSION} cosbench-driver-web_${VERSION}" + + OSGI_CONSOLE_PORT=18089 + diff --git a/dev/cosbench-manta/docker_build/usr/local/bin/proclimit b/dev/cosbench-manta/docker_build/usr/local/bin/proclimit new file mode 100644 index 00000000..d799f2d8 --- /dev/null +++ b/dev/cosbench-manta/docker_build/usr/local/bin/proclimit @@ -0,0 +1,42 @@ +#!/usr/bin/env bash + +# Just return the value from nproc if we are not on a Triton container +if [ ! -d /native ]; then + nproc + exit 0 +fi + +## +# Returns a number representing a very conservative estimate of the maximum +# number of processes or threads that you want to run within the zone that +# invoked this script. Typically, you would take this value and define a +# multiplier that works well for your application. + +if [[ -d /native ]]; then + PATH=$PATH:/native/usr/bin +fi + +set -o errexit +if [[ -n ${TRACE} ]]; then + set -o xtrace +fi + +# CN parameters +CORES=$(kstat -C -m cpu_info -c misc -s core_id | wc -l | tr -d ' ') +PHYS_MEM=$(kstat -C -m unix -n system_pages -c pages -s physmem | cut -d':' -f5) +PAGESIZE=$(pagesize) +TOTAL_MEMORY=$(echo "${PHYS_MEM} ${PAGESIZE} * pq" | dc) + +# zone parameters +ZONE_MEMORY=$(kstat -C -m memory_cap -c zone_memory_cap -s physcap | cut -d':' -f5) + +# our fraction of the total memory on the CN +MEMORY_SHARE=$(echo "8k$ZONE_MEMORY $TOTAL_MEMORY / pq" | dc) + +# best guess as to how many CPUs we should pretend like we have for tuning +CPU_GUESS=$(echo "${CORES} ${MEMORY_SHARE} * pq" | dc) + +# round that up to a positive integer +echo ${CPU_GUESS} | awk 'function ceil(valor) { return (valor == int(valor) && value != 0) ? valor : int(valor)+1 } { printf "%d", ceil($1) }' + +exit 0 diff --git a/dev/cosbench-manta/pom.xml b/dev/cosbench-manta/pom.xml new file mode 100644 index 00000000..39dab74d --- /dev/null +++ b/dev/cosbench-manta/pom.xml @@ -0,0 +1,442 @@ + + + 4.0.0 + + com.joyent.manta + cosbench-manta + 1.0.2-SNAPSHOT + bundle + + cosbench-manta + Manta adapter for Cosbench + https://github.com/joyent/manta-cosbench + 2015 + + Joyent + joyent.com + + + + The MIT License (MIT) + http://opensource.org/licenses/MIT + repo + Copyright (c) 2015 Joyent + + + + + + Elijah Zupancic + elijah@zupancic.name + Joyent + joyent.com + + + + + scm:git:git://github.com/joyent/cosbench-manta.git + scm:git:git://github.com/joyent/cosbench-manta.git + git@github.com:joyent/cosbench-manta.git + HEAD + + + github.com + https://github.com/joyent/cosbench-manta/issues + + + + ossrh + https://oss.sonatype.org/content/repositories/snapshots + + + ossrh + https://oss.sonatype.org/service/local/staging/deploy/maven2/ + + + + + 1.8 + 3.2 + false + UTF-8 + UTF-8 + false + true + 1 + + 3.3 + 3.0.1 + 1.4.1 + 1.0-beta-4 + 2.17 + 2.5 + 3.0.0 + 3.3 + 2.10 + 2.8.2 + 1.4.1 + 1.0-beta-4 + 1.6 + 2.5.2 + 2.6 + 1.4 + 2.10.3 + 2.5 + 2.5.3 + 2.7 + 2.4 + 2.19 + + 6.12.1 + 2.0.0 + 1.7.7 + 1.0 + 0.4.1.0 + + + + + + project.local + project + file:${project.basedir}/build-dependencies + + + + + + com.intel.cosbench + cosbench-api + ${dependency.cosbench.version} + provided + + + com.intel.cosbench + cosbench-config + ${dependency.cosbench.version} + provided + + + com.intel.cosbench + cosbench-log + ${dependency.cosbench.version} + provided + + + + com.joyent.manta + java-manta-client + ${dependency.java-manta.version} + + + org.slf4j + slf4j-log4j12 + ${dependency.slf4j-log4j12.version} + + + log4j + log4j + + + + + log4j + log4j + [1.0.0,2.0.0) + provided + + + org.osgi + osgi_R4_core + ${dependency.osgi4.version} + provided + true + + + + + + + + org.apache.felix + maven-bundle-plugin + ${maven-bundle-plugin.version} + true + + + Manta Storage Client Bundle + Joyent + cosbench-manta + com.joyent.manta.cosbench.Activator + com.joyent.manta.cosbench.config.* + + *;scope=compile|runtime;artifactId=!log4j + true + {maven-resources} + + + osgi.ee;filter:="(&(osgi.ee=JavaSE))" + + + + + + + org.apache.maven.plugins + maven-checkstyle-plugin + ${maven-checkstyle-plugin.version} + + + com.puppycrawl.tools + checkstyle + ${dependency.checkstyle.version} + + + + checkstyle.xml + true + ${checkstyle.skip} + suppressions.xml + warning + + + + checkstyle + validate + + check + + + + + + + + org.apache.maven.plugins + maven-enforcer-plugin + ${maven-enforcer-plugin.version} + + + enforce-maven + + enforce + + + + + [${maven.min.version},) + + + + + + enforce + + + + + + + enforce + + + + enforce-bytecode-version + + enforce + + + + + ${java.min.version} + + + true + + + + enforce-ban-circular-dependencies + + enforce + + + + + + true + + + + enforce-ban-duplicate-classes + + enforce + + + + + true + + + true + + + + + + org.codehaus.mojo + extra-enforcer-rules + ${maven-extra-enforcer-rules.version} + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + ${maven-compiler-plugin.version} + + ${java.min.version} + ${java.min.version} + + + + org.apache.maven.plugins + maven-surefire-plugin + ${maven-surefire-plugin.version} + + ${surefire.forkCount} + ${maven.test.skip} + + + ${surefire.useSystemClassLoader} + + + + org.apache.maven.plugins + maven-clean-plugin + ${maven-clean-plugin.version} + + + org.apache.maven.plugins + maven-dependency-plugin + ${maven-dependency-plugin.version} + + + org.apache.maven.plugins + maven-deploy-plugin + ${maven-deploy-plugin.version} + + + org.apache.maven.plugins + maven-gpg-plugin + ${maven-gpg-plugin.version} + + + org.apache.maven.plugins + maven-install-plugin + ${maven-install-plugin.version} + + + org.apache.maven.plugins + maven-jar-plugin + ${maven-jar-plugin.version} + + + org.apache.maven.plugins + maven-jarsigner-plugin + ${maven-jarsigner-plugin.version} + + + org.apache.maven.plugins + maven-javadoc-plugin + ${maven-javadoc-plugin.version} + + + org.apache.maven.plugins + maven-release-plugin + ${maven-release-plugin.version} + + true + false + release + + + + org.apache.maven.plugins + maven-resources-plugin + ${maven-resources-plugin.version} + + + org.apache.maven.plugins + maven-source-plugin + ${maven-source-plugin.version} + + + + + + + + org.apache.maven.plugins + maven-jxr-plugin + ${maven-jxr-plugin.version} + + + + + + + + release + + + + org.apache.maven.plugins + maven-gpg-plugin + + + sign-artifacts + verify + + sign + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + + + attach-javadocs + + jar + + + + + + org.apache.maven.plugins + maven-source-plugin + + + attach-sources + + jar-no-fork + + + + + + + + + diff --git a/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/Activator.java b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/Activator.java new file mode 100644 index 00000000..caf96fb0 --- /dev/null +++ b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/Activator.java @@ -0,0 +1,50 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.joyent.manta.cosbench; + +import com.intel.cosbench.log.LogFactory; +import org.osgi.framework.BundleActivator; +import org.osgi.framework.BundleContext; + +/** + * Default OSGI activator that provides the startup and shutdown functionaly + * needed to load and unload classes from OSGI. + * + * @author Elijah Zupancic + */ +public class Activator implements BundleActivator { + + /** + * OSGI start up method. + * @param bundleContext OSGI bundle + * @throws Exception when there is a problem in OSGI start up + */ + public void start(final BundleContext bundleContext) throws Exception { + LogFactory.getSystemLogger().info("Starting Manta adapter"); + } + + /** + * OSGI stop method. + * @param bundleContext OSGI bundle + * @throws Exception when there is a problem in OSGI shut down + */ + public void stop(final BundleContext bundleContext) throws Exception { + LogFactory.getSystemLogger().info("Stopping Manta adapter"); + } +} diff --git a/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/MantaStorage.java b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/MantaStorage.java new file mode 100644 index 00000000..31827ffd --- /dev/null +++ b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/MantaStorage.java @@ -0,0 +1,290 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.joyent.manta.cosbench; + +import com.joyent.manta.cosbench.config.CosbenchMantaConfigContext; +import com.intel.cosbench.api.storage.NoneStorage; +import com.intel.cosbench.api.storage.StorageException; +import com.intel.cosbench.config.Config; +import com.intel.cosbench.log.Logger; +import com.joyent.manta.client.MantaClient; +import com.joyent.manta.client.MantaHttpHeaders; +import com.joyent.manta.client.MantaMetadata; +import com.joyent.manta.config.ChainedConfigContext; +import com.joyent.manta.config.DefaultsConfigContext; +import com.joyent.manta.config.EnvVarConfigContext; +import com.joyent.manta.config.SystemSettingsConfigContext; +import com.joyent.manta.exception.MantaClientHttpResponseException; +import com.joyent.manta.exception.MantaErrorCode; + +import java.io.IOException; +import java.io.InputStream; +import java.util.HashMap; +import java.util.Map; + +/** + * Manta implementation of the COSBench {@link com.intel.cosbench.api.storage.StorageAPI}. + * + * @author Elijah Zupancic + */ +public class MantaStorage extends NoneStorage { + /** + * Hardcoded directory in Manta in which all benchmark files are stored. + */ + private static final String COSBENCH_BASE_DIR = "stor/cosbench"; + + /** + * Manta client driver. + */ + private MantaClient client; + + /** + * The current test directory name. + */ + private String currentTestDirectory; + + /** + * Flag indicating whether or not to send files using chunked encoding. + */ + private boolean chunked = false; + + @Override + public void init(final Config config, final Logger logger) { + logger.debug("Manta client has started initialization"); + super.init(config, logger); + + final ChainedConfigContext context = new ChainedConfigContext( + new DefaultsConfigContext(), + new EnvVarConfigContext(), + new SystemSettingsConfigContext(), + new CosbenchMantaConfigContext(config)); + + if (logger.isDebugEnabled()) { + String msg = String.format("Configuration: [user=%s, key_path=%s, url=%s]", + context.getMantaUser(), context.getMantaKeyPath(), + context.getMantaURL()); + logger.debug(msg); + } + + try { + this.chunked = config.getBoolean("chunked"); + } catch (RuntimeException e) { + logger.warn("Couldn't get value of chunked configuration setting"); + } + + try { + client = new MantaClient(context); + + // We rely on COSBench properly cleaning up after itself. + currentTestDirectory = String.format("%s/%s", + context.getMantaHomeDirectory(), COSBENCH_BASE_DIR); + + client.putDirectory(currentTestDirectory, true); + + if (!client.existsAndIsAccessible(currentTestDirectory)) { + String msg = "Unable to create base test directory"; + throw new StorageException(msg); + } + + } catch (IOException e) { + logger.error("Error in initialization", e); + throw new StorageException(e); + } + + logger.debug("Manta client has been initialized"); + } + + @Override + public void createContainer(final String container, final Config config) { + super.createContainer(container, config); + + try { + client.putDirectory(directoryOfContainer(container)); + } catch (IOException e) { + logger.error("Error creating container", e); + throw new StorageException(e); + } + } + + @Override + public void deleteContainer(final String container, final Config config) { + super.deleteContainer(container, config); + + try { + client.deleteRecursive(directoryOfContainer(container)); + } catch (MantaClientHttpResponseException e) { + if (!e.getServerCode().equals(MantaErrorCode.RESOURCE_NOT_FOUND_ERROR)) { + logger.error("Error error deleting object", e); + throw new StorageException(e); + } + } catch (IOException e) { + logger.error("Error deleting container", e); + throw new StorageException(e); + } + } + + @Override + public void createObject(final String container, + final String object, + final InputStream data, + final long length, + final Config config) { + super.createObject(container, object, data, length, config); + + final String path = pathOfObject(container, object); + + try { + MantaHttpHeaders headers = new MantaHttpHeaders(); + + if (chunked) { + headers.setContentEncoding("chunked"); + } + + client.put(path, data, headers); + } catch (MantaClientHttpResponseException e) { + // This is a fall-back in the weird cases where COSBench doesn't + // do things in the right order. + if (e.getServerCode().equals(MantaErrorCode.DIRECTORY_DOES_NOT_EXIST_ERROR)) { + try { + String dir = directoryOfContainer(container); + client.putDirectory(dir, true); + client.put(path, data); + } catch (IOException ioe) { + throw new StorageException(ioe); + } + } else { + throw new StorageException(e); + } + } catch (IOException e) { + logger.error("Error error creating object", e); + throw new StorageException(e); + } + } + + @Override + public void deleteObject(final String container, final String object, + final Config config) { + super.deleteObject(container, object, config); + + try { + String path = pathOfObject(container, object); + client.delete(path); + } catch (MantaClientHttpResponseException e) { + if (!e.getServerCode().equals(MantaErrorCode.RESOURCE_NOT_FOUND_ERROR)) { + logger.error("Error error deleting object", e); + throw new StorageException(e); + } + } catch (IOException e) { + logger.error("Error error deleting object", e); + throw new StorageException(e); + } + } + + @Override + public InputStream getObject(final String container, final String object, final Config config) { + super.getObject(container, object, config); + + try { + final String path = pathOfObject(container, object); + return client.getAsInputStream(path); + } catch (IOException e) { + logger.error("Error error getting object", e); + throw new StorageException(e); + } + } + + @Override + protected void createMetadata(final String container, + final String object, + final Map map, + final Config config) { + super.createMetadata(container, object, map, config); + + try { + String path = pathOfObject(container, object); + Map prefixedMap = new HashMap<>(map.size()); + String format = "m-%s"; + + for (Map.Entry entry : map.entrySet()) { + prefixedMap.put(String.format(format, entry.getKey()), entry.getValue()); + } + + MantaMetadata metadata = new MantaMetadata(prefixedMap); + client.putMetadata(path, metadata); + } catch (IOException e) { + logger.error("Error error creating metadata", e); + throw new StorageException(e); + } + } + + @Override + protected Map getMetadata(final String container, + final String object, + final Config config) { + super.getMetadata(container, object, config); + + try { + String path = pathOfObject(container, object); + return client.head(path).getMetadata(); + } catch (IOException e) { + logger.error("Error error getting metadata", e); + throw new StorageException(e); + } + } + + @Override + public void dispose() { + super.dispose(); + + try { + if (client != null) { + client.close(); + } + } catch (Exception e) { + logger.warn("Error when attempting to close Manta client", e); + } + + client = null; + } + + @Override + public void abort() { + super.abort(); + } + + /** + * Utility method that provides the directory mapping of a container. + * @param container container name + * @return directory as string + */ + private String directoryOfContainer(final String container) { + return String.format("%s/%s", currentTestDirectory, container); + } + + /** + * Utility method that provides the directory mapping of an object. + * @param container container name + * @param object object name + * @return full path to object as string + */ + private String pathOfObject(final String container, final String object) { + String dir = directoryOfContainer(container); + return String.format("%s/%s", dir, object); + } +} diff --git a/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/MantaStorageFactory.java b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/MantaStorageFactory.java new file mode 100644 index 00000000..d4a8050c --- /dev/null +++ b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/MantaStorageFactory.java @@ -0,0 +1,43 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.joyent.manta.cosbench; + +import com.intel.cosbench.api.storage.StorageAPI; +import com.intel.cosbench.api.storage.StorageAPIFactory; + +/** + * {@link StorageAPIFactory} implementation that defines the properties + * of the Manta adaptor. + * + * @author Elijah Zupancic + */ +public class MantaStorageFactory implements StorageAPIFactory { + + @Override + public String getStorageName() { + return "manta"; + } + + @Override + public StorageAPI getStorageAPI() { + return new MantaStorage(); + } + +} + diff --git a/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/config/CosbenchMantaConfigContext.java b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/config/CosbenchMantaConfigContext.java new file mode 100644 index 00000000..dac7d36e --- /dev/null +++ b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/config/CosbenchMantaConfigContext.java @@ -0,0 +1,148 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +package com.joyent.manta.cosbench.config; + +import com.intel.cosbench.config.Config; +import com.intel.cosbench.log.LogFactory; +import com.intel.cosbench.log.Logger; +import com.joyent.manta.config.ConfigContext; + +/** + * Cosbench specific implementation of {@link ConfigContext} that allows us + * to connect Cosbench config seamlessly. + * + * @author Elijah Zupancic + */ +public class CosbenchMantaConfigContext implements ConfigContext { + /** + * Logger instance. + */ + private static Logger logger = LogFactory.getSystemLogger(); + + /** + * Embedded Cosbench config reference. + */ + private final Config config; + + /** + * Default constructor that wraps a Cosbench config instance. + * @param config cosbench config instance + */ + public CosbenchMantaConfigContext(final Config config) { + this.config = config; + } + + @Override + public String getMantaURL() { + // COSBench throws errors when it doesn't have a config key. + // This is our hacky workaround. + try { + return config.get("url"); + } catch (RuntimeException e) { + logger.trace("Couldn't get url from COSBench config", e); + return null; + } + } + + @Override + public String getMantaUser() { + try { + return config.get("username"); + } catch (RuntimeException e) { + logger.trace("Couldn't get username from COSBench config", e); + return null; + } + } + + @Override + public String getMantaKeyId() { + try { + return config.get("fingerprint"); + } catch (RuntimeException e) { + logger.trace("Couldn't get fingerprint from COSBench config", e); + return null; + } + } + + @Override + public String getMantaKeyPath() { + try { + return config.get("key_path"); + } catch (RuntimeException e) { + logger.trace("Couldn't get key_path from COSBench config", e); + return null; + } + } + + @Override + public String getPrivateKeyContent() { + // We don't support embedded key content with COSBench + return null; + } + + @Override + public String getPassword() { + try { + return config.get("password"); + } catch (RuntimeException e) { + logger.trace("Couldn't get password from COSBench config", e); + return null; + } + } + + @Override + public Integer getTimeout() { + return safeGetInteger( + "timeout", "Couldn't get timeout from COSBench config"); + } + + @Override + public Integer getRetries() { + return safeGetInteger( + "retries", "Couldn't get retries from COSBench config"); + } + + /** + * Utility method that checks for the presence of Integer values in + * the COSBench configuration and then returns the value if found. + * @param key key to check for + * @param message message to display when value isn't present + * @return null if not found, otherwise configuration value + */ + private Integer safeGetInteger(final String key, final String message) { + try { + int configValue = config.getInt(key, Integer.MIN_VALUE); + + if (configValue != Integer.MIN_VALUE) { + return configValue; + } else { + return null; + } + + } catch (RuntimeException e) { + logger.trace(message, e); + return null; + } + } + + @Override + public String getMantaHomeDirectory() { + return ConfigContext.deriveHomeDirectoryFromUser(getMantaUser()); + } +} diff --git a/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/config/package-info.java b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/config/package-info.java new file mode 100644 index 00000000..2415ffa8 --- /dev/null +++ b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/config/package-info.java @@ -0,0 +1,24 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** + * Package containing configuration classes. + * + * @author Elijah Zupancic + */ +package com.joyent.manta.cosbench.config; diff --git a/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/package-info.java b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/package-info.java new file mode 100644 index 00000000..4c2a4674 --- /dev/null +++ b/dev/cosbench-manta/src/main/java/com/joyent/manta/cosbench/package-info.java @@ -0,0 +1,25 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** + * Main package containing the Manta implementation of a COSBench + * adapter. + * + * @author Elijah Zupancic + */ +package com.joyent.manta.cosbench; diff --git a/dev/cosbench-manta/src/main/resources/META-INF/spring/plugin-context.xml b/dev/cosbench-manta/src/main/resources/META-INF/spring/plugin-context.xml new file mode 100644 index 00000000..0a052056 --- /dev/null +++ b/dev/cosbench-manta/src/main/resources/META-INF/spring/plugin-context.xml @@ -0,0 +1,17 @@ + + + + + + + + + + diff --git a/dev/cosbench-manta/suppressions.xml b/dev/cosbench-manta/suppressions.xml new file mode 100644 index 00000000..b209b107 --- /dev/null +++ b/dev/cosbench-manta/suppressions.xml @@ -0,0 +1,13 @@ + + + + + + + + + + + + + diff --git a/dev/cosbench-mock/META-INF/MANIFEST.MF b/dev/cosbench-mock/META-INF/MANIFEST.MF index 8598971e..1e2aeada 100644 --- a/dev/cosbench-mock/META-INF/MANIFEST.MF +++ b/dev/cosbench-mock/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Mock Client Bundle Bundle-SymbolicName: cosbench-mock -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockAuth.java b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockAuth.java index cc195a38..41885dd2 100644 --- a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockAuth.java +++ b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockAuth.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.mock; @@ -28,9 +28,9 @@ /** * This is an mocked authentication mechanism, which just inserts a short delay * before return for login request. - * + * * @author ywang19, qzheng7 - * + * */ class MockAuth extends NoneAuth { @@ -53,7 +53,7 @@ public void init(Config config, Logger logger) { parms.put(OP_DELAY_KEY, delay); logger.debug("using auth config: {}", parms); - + logger.debug("mock auth has been initialized"); } diff --git a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockAuthFactory.java b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockAuthFactory.java index 6f0743b8..2daeedef 100644 --- a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockAuthFactory.java +++ b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockAuthFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.mock; diff --git a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockConstants.java b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockConstants.java index de427584..4923fc03 100644 --- a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockConstants.java +++ b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockConstants.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.mock; diff --git a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockStorage.java b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockStorage.java index 19bf0100..2cb0ba5c 100644 --- a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockStorage.java +++ b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockStorage.java @@ -1,182 +1,182 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.api.mock; - -import static com.intel.cosbench.api.mock.MockConstants.*; - -import java.io.InputStream; -import java.util.Random; - -import org.apache.commons.io.input.NullInputStream; - -import com.intel.cosbench.api.storage.*; -import com.intel.cosbench.config.Config; -import com.intel.cosbench.log.Logger; - -/** - * This is an mocked storage, which just inserts a short delay before return for - * each request. - * - * @author ywang19, qzheng7 - * - */ -class MockStorage extends NoneStorage { - - private Random random = new Random(); - private Statistics stats = new Statistics(); - - /* configurations */ - private long size; // object size (in bytes) - private long delay; // operation delay (in milliseconds) - private double errors; // error rate for error injection [0,1] - private boolean printing; // enable printing object content (for debugging) - private boolean profiling; // enable operation profiling (for debugging) - - /* current working thread */ - private volatile Thread thread; - - public MockStorage() { - /* empty */ - } - - @Override - public void init(Config config, Logger logger) { - super.init(config, logger); - - stats.addProfile("GET"); - stats.addProfile("PUT"); - stats.addProfile("DEL"); - logger.debug("op profiling data has been initialized"); - - size = config.getLong(OBJECT_SIZE_KEY, OBJECT_SIZE_DEFAULT); - delay = config.getLong(OP_DELAY_KEY, OP_DELAY_DEFAULT); - errors = config.getDouble(OP_ERRORS_KEY, OP_ERRORS_DEFAULT); - printing = config.getBoolean(PRINTING_KEY, PRINTING_DEFAULT); - profiling = config.getBoolean(PROFILING_KEY, PROFILING_DEFAULT); - - parms.put(OBJECT_SIZE_KEY, size); - parms.put(OP_DELAY_KEY, delay); - parms.put(OP_ERRORS_KEY, errors); - parms.put(PRINTING_KEY, printing); - parms.put(PROFILING_KEY, profiling); - - logger.debug("using storage config: {}", parms); - - logger.debug("mock client has been initialized"); - } - - @Override - public void dispose() { - super.dispose(); - if (profiling && logger.isDebugEnabled()) - stats.printStats(logger); - thread = null; - } - - @Override - public void abort() { - super.abort(); - if (thread != null) - thread.interrupt(); - thread = null; - } - - @Override - public InputStream getObject(String container, String object, Config config) { - super.getObject(container, object, config); - if (profiling && logger.isDebugEnabled()) - stats.addEvent("GET", container + "/" + object); - if (random.nextDouble() < errors) - throw new StorageException("error injection"); - thread = Thread.currentThread(); - MockUtils.sleep(delay); - return new NullInputStream(size); - } - - @Override - public InputStream getList(String container, String object, Config config) { - super.getList(container, object, config); - if (profiling && logger.isDebugEnabled()) - stats.addEvent("LIST", container + "/" + object); //### - if (random.nextDouble() < errors) - throw new StorageException("error injection"); - thread = Thread.currentThread(); - MockUtils.sleep(delay); - return new NullInputStream(0); //### - } - - @Override - public void createContainer(String container, Config config) { - super.createContainer(container, config); - if (random.nextDouble() < errors) - throw new StorageException("error injection"); - thread = Thread.currentThread(); - MockUtils.sleep(delay); - } - - @Deprecated - public void createObject(String container, String object, byte[] data, - Config config) { - super.createObject(container, object, data, config); - if (profiling && logger.isDebugEnabled()) - stats.addEvent("PUT", container + "/" + object); - if (random.nextDouble() < errors) - throw new StorageException("error injection"); - if (printing) - logger.debug("content to upload: " + MockUtils.toString(data)); - thread = Thread.currentThread(); - MockUtils.sleep(delay); - } - - @Override - public void createObject(String container, String object, InputStream data, - long length, Config config) { - super.createObject(container, object, data, length, config); - if (profiling && logger.isDebugEnabled()) - stats.addEvent("PUT", container + "/" + object); - if (random.nextDouble() < errors) - throw new StorageException("error injection"); - if (printing) - logger.debug("content to upload: " + MockUtils.toString(data)); - else - MockUtils.consume(data); - thread = Thread.currentThread(); - MockUtils.sleep(delay); - } - - @Override - public void deleteContainer(String container, Config config) { - super.deleteContainer(container, config); - if (random.nextDouble() < errors) - throw new StorageException("error injection"); - thread = Thread.currentThread(); - MockUtils.sleep(delay); - } - - @Override - public void deleteObject(String container, String object, Config config) { - super.deleteObject(container, object, config); - if (profiling && logger.isDebugEnabled()) - stats.addEvent("DEL", container + "/" + object); - if (random.nextDouble() < errors) - throw new StorageException("error injection"); - thread = Thread.currentThread(); - MockUtils.sleep(delay); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.api.mock; + +import static com.intel.cosbench.api.mock.MockConstants.*; + +import java.io.InputStream; +import java.util.Random; + +import org.apache.commons.io.input.NullInputStream; + +import com.intel.cosbench.api.storage.*; +import com.intel.cosbench.config.Config; +import com.intel.cosbench.log.Logger; + +/** + * This is an mocked storage, which just inserts a short delay before return for + * each request. + * + * @author ywang19, qzheng7 + * + */ +class MockStorage extends NoneStorage { + + private Random random = new Random(); + private Statistics stats = new Statistics(); + + /* configurations */ + private long size; // object size (in bytes) + private long delay; // operation delay (in milliseconds) + private double errors; // error rate for error injection [0,1] + private boolean printing; // enable printing object content (for debugging) + private boolean profiling; // enable operation profiling (for debugging) + + /* current working thread */ + private volatile Thread thread; + + public MockStorage() { + /* empty */ + } + + @Override + public void init(Config config, Logger logger) { + super.init(config, logger); + + stats.addProfile("GET"); + stats.addProfile("PUT"); + stats.addProfile("DEL"); + logger.debug("op profiling data has been initialized"); + + size = config.getLong(OBJECT_SIZE_KEY, OBJECT_SIZE_DEFAULT); + delay = config.getLong(OP_DELAY_KEY, OP_DELAY_DEFAULT); + errors = config.getDouble(OP_ERRORS_KEY, OP_ERRORS_DEFAULT); + printing = config.getBoolean(PRINTING_KEY, PRINTING_DEFAULT); + profiling = config.getBoolean(PROFILING_KEY, PROFILING_DEFAULT); + + parms.put(OBJECT_SIZE_KEY, size); + parms.put(OP_DELAY_KEY, delay); + parms.put(OP_ERRORS_KEY, errors); + parms.put(PRINTING_KEY, printing); + parms.put(PROFILING_KEY, profiling); + + logger.debug("using storage config: {}", parms); + + logger.debug("mock client has been initialized"); + } + + @Override + public void dispose() { + super.dispose(); + if (profiling && logger.isDebugEnabled()) + stats.printStats(logger); + thread = null; + } + + @Override + public void abort() { + super.abort(); + if (thread != null) + thread.interrupt(); + thread = null; + } + + @Override + public InputStream getObject(String container, String object, Config config) { + super.getObject(container, object, config); + if (profiling && logger.isDebugEnabled()) + stats.addEvent("GET", container + "/" + object); + if (random.nextDouble() < errors) + throw new StorageException("error injection"); + thread = Thread.currentThread(); + MockUtils.sleep(delay); + return new NullInputStream(size); + } + + @Override + public InputStream getList(String container, String object, Config config) { + super.getList(container, object, config); + if (profiling && logger.isDebugEnabled()) + stats.addEvent("LIST", container + "/" + object); //### + if (random.nextDouble() < errors) + throw new StorageException("error injection"); + thread = Thread.currentThread(); + MockUtils.sleep(delay); + return new NullInputStream(0); //### + } + + @Override + public void createContainer(String container, Config config) { + super.createContainer(container, config); + if (random.nextDouble() < errors) + throw new StorageException("error injection"); + thread = Thread.currentThread(); + MockUtils.sleep(delay); + } + + @Deprecated + public void createObject(String container, String object, byte[] data, + Config config) { + super.createObject(container, object, data, config); + if (profiling && logger.isDebugEnabled()) + stats.addEvent("PUT", container + "/" + object); + if (random.nextDouble() < errors) + throw new StorageException("error injection"); + if (printing) + logger.debug("content to upload: " + MockUtils.toString(data)); + thread = Thread.currentThread(); + MockUtils.sleep(delay); + } + + @Override + public void createObject(String container, String object, InputStream data, + long length, Config config) { + super.createObject(container, object, data, length, config); + if (profiling && logger.isDebugEnabled()) + stats.addEvent("PUT", container + "/" + object); + if (random.nextDouble() < errors) + throw new StorageException("error injection"); + if (printing) + logger.debug("content to upload: " + MockUtils.toString(data)); + else + MockUtils.consume(data); + thread = Thread.currentThread(); + MockUtils.sleep(delay); + } + + @Override + public void deleteContainer(String container, Config config) { + super.deleteContainer(container, config); + if (random.nextDouble() < errors) + throw new StorageException("error injection"); + thread = Thread.currentThread(); + MockUtils.sleep(delay); + } + + @Override + public void deleteObject(String container, String object, Config config) { + super.deleteObject(container, object, config); + if (profiling && logger.isDebugEnabled()) + stats.addEvent("DEL", container + "/" + object); + if (random.nextDouble() < errors) + throw new StorageException("error injection"); + thread = Thread.currentThread(); + MockUtils.sleep(delay); + } + +} diff --git a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockStorageFactory.java b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockStorageFactory.java index c4517363..4e5c6580 100644 --- a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockStorageFactory.java +++ b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockStorageFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.mock; diff --git a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockUtils.java b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockUtils.java index d3bc65c7..c4bc2aec 100644 --- a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockUtils.java +++ b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/MockUtils.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.mock; diff --git a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/Profile.java b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/Profile.java index 9029a128..baa8850a 100644 --- a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/Profile.java +++ b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/Profile.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.mock; diff --git a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/Statistics.java b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/Statistics.java index 2ac20a25..12efcde4 100644 --- a/dev/cosbench-mock/src/com/intel/cosbench/api/mock/Statistics.java +++ b/dev/cosbench-mock/src/com/intel/cosbench/api/mock/Statistics.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.mock; diff --git a/dev/cosbench-openio/.classpath b/dev/cosbench-openio/.classpath index 72649e55..db4fc35a 100644 --- a/dev/cosbench-openio/.classpath +++ b/dev/cosbench-openio/.classpath @@ -1,10 +1,9 @@ - - + diff --git a/dev/cosbench-openio/META-INF/MANIFEST.MF b/dev/cosbench-openio/META-INF/MANIFEST.MF index 5e5431fa..ec3c72d6 100644 --- a/dev/cosbench-openio/META-INF/MANIFEST.MF +++ b/dev/cosbench-openio/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: OpenIO Storage Client Bundle Bundle-SymbolicName: cosbench-openio -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: Intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, @@ -23,5 +23,5 @@ Import-Package: com.intel.cosbench.api.auth, org.apache.http.util;version="[4.1.4,5.0.0)" Bundle-ClassPath: ., gson-2.5.jar, - openio-api-0.3.2-SNAPSHOT.jar + openio-api-1.1.0.jar Export-Package: com.intel.cosbench.api.openio diff --git a/dev/cosbench-openio/README.md b/dev/cosbench-openio/README.md new file mode 100644 index 00000000..e879c0b6 --- /dev/null +++ b/dev/cosbench-openio/README.md @@ -0,0 +1,30 @@ +# OpenIO COSBench plugin + +The plugin implements basic storage functions (create/delete containers and create/get/ delete objects). + +## Configuration + +Elements to set in workload config parameter: + +* `ns`: name of the namespace +* `account`: the account in which the plugin will work +* `proxyd-url`: the URL of the OpenIO proxyd service of your namespace +* *for erasure coding* `ecd-url`: URL of an ECD service + +See workload example [here](../../release/conf/openio-config-sample.xml). + + +## Updating the plugin + +This only works for stable versions. Snapshot versions of openio-api cannot be +downloaded with maven's dependency:copy plugin (unless you are a maven expert). + +```bash +PREV=$((cd dev/cosbench-openio && /bin/ls -1 -v openio-api-* ) | tail -n 1) +mvn dependency:copy -Dartifact=io.openio.sds:openio-api:LATEST -DoutputDirectory=dev/cosbench-openio/ +LATEST=$((cd dev/cosbench-openio && /bin/ls -1 -v openio-api-* ) | tail -n 1) +sed -i -e "s/openio-api-.*\.jar/${LATEST}/g" dev/cosbench-openio/META-INF/MANIFEST.MF dev/cosbench-openio/.classpath dev/cosbench-openio/build.properties +git rm "dev/cosbench-openio/${PREV}" +git add "dev/cosbench-openio/${LATEST}" +git add dev/cosbench-openio/META-INF/MANIFEST.MF dev/cosbench-openio/.classpath dev/cosbench-openio/build.properties +``` diff --git a/dev/cosbench-openio/REAME.md b/dev/cosbench-openio/REAME.md deleted file mode 100644 index a6fd015d..00000000 --- a/dev/cosbench-openio/REAME.md +++ /dev/null @@ -1,8 +0,0 @@ -

OpenIO COSBench plugin

- -

Required elements to set in Config object: -

    -
  • "account" : the account in which the plugin will work
    • -
  • "proxyd-url" : the url of the OpenIO proxyd service of your namespace
    • -
-

The plugin implements basis storage functions (create/delete containers and create / get / delete objects).

\ No newline at end of file diff --git a/dev/cosbench-openio/build.properties b/dev/cosbench-openio/build.properties index 6c2ae212..9225b2a3 100644 --- a/dev/cosbench-openio/build.properties +++ b/dev/cosbench-openio/build.properties @@ -3,4 +3,4 @@ output.. = bin/ bin.includes = META-INF/,\ .,\ gson-2.5.jar,\ - openio-api-0.3.2-SNAPSHOT.jar + openio-api-1.1.0.jar diff --git a/dev/cosbench-openio/openio-api-0.3.2-SNAPSHOT.jar b/dev/cosbench-openio/openio-api-0.3.2-SNAPSHOT.jar deleted file mode 100644 index 0cbf5dcd..00000000 Binary files a/dev/cosbench-openio/openio-api-0.3.2-SNAPSHOT.jar and /dev/null differ diff --git a/dev/cosbench-openio/openio-api-1.1.0.jar b/dev/cosbench-openio/openio-api-1.1.0.jar new file mode 100644 index 00000000..386c4b2a Binary files /dev/null and b/dev/cosbench-openio/openio-api-1.1.0.jar differ diff --git a/dev/cosbench-openio/src/com/intel/cosbench/api/openio/OpenIOStorage.java b/dev/cosbench-openio/src/com/intel/cosbench/api/openio/OpenIOStorage.java index dc318458..002b25d0 100644 --- a/dev/cosbench-openio/src/com/intel/cosbench/api/openio/OpenIOStorage.java +++ b/dev/cosbench-openio/src/com/intel/cosbench/api/openio/OpenIOStorage.java @@ -1,8 +1,27 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.openio; import static com.intel.cosbench.client.openio.OioStorageConstants.ACCOUNT_KEY; import static com.intel.cosbench.client.openio.OioStorageConstants.NS_KEY; import static com.intel.cosbench.client.openio.OioStorageConstants.PROXYD_URL_KEY; +import static com.intel.cosbench.client.openio.OioStorageConstants.ECD_URL_KEY; import static io.openio.sds.models.OioUrl.url; import java.io.ByteArrayInputStream; @@ -19,6 +38,9 @@ import io.openio.sds.ClientBuilder; import io.openio.sds.exceptions.ContainerExistException; import io.openio.sds.exceptions.ContainerNotEmptyException; +import io.openio.sds.exceptions.ContainerNotFoundException; +import io.openio.sds.exceptions.DeadlineReachedException; +import io.openio.sds.exceptions.ObjectNotFoundException; import io.openio.sds.exceptions.SdsException; import io.openio.sds.models.ListOptions; import io.openio.sds.models.ObjectInfo; @@ -26,7 +48,7 @@ import io.openio.sds.models.ObjectList.ObjectView; /** - * + * * @author Christopher Dedeurwaerder * */ @@ -41,8 +63,9 @@ public void init(Config config, Logger logger) { super.init(config, logger); String ns = config.get(NS_KEY); String proxydurl = config.get(PROXYD_URL_KEY); + String ecdUrl = config.get(ECD_URL_KEY, "http://localhost:6001"); client = ClientBuilder - .newClient(ns, proxydurl); + .newClient(ns, proxydurl, ecdUrl); account = config.get(ACCOUNT_KEY); parms.put(NS_KEY, ns); parms.put(PROXYD_URL_KEY, proxydurl); @@ -60,8 +83,14 @@ public void init(Config config, Logger logger) { private StorageException makeStorageException(SdsException wrapped) { String origMsg = wrapped.getMessage(); if (wrapped instanceof ContainerNotEmptyException - || wrapped instanceof ContainerExistException) + || wrapped instanceof ContainerExistException) { return new StorageException("HTTP/1.1 409 " + origMsg, wrapped); + } else if (wrapped instanceof ObjectNotFoundException || + wrapped instanceof ContainerNotFoundException) { + return new StorageException("HTTP/1.1 404 " + origMsg, wrapped); + } else if (wrapped instanceof DeadlineReachedException) { + return new StorageException("HTTP/1.1 503 " + origMsg, wrapped); + } Matcher codeMatcher = errCodePattern.matcher(origMsg); if (codeMatcher.find()) { diff --git a/dev/cosbench-openio/src/com/intel/cosbench/api/openio/OpenIOStorageFactory.java b/dev/cosbench-openio/src/com/intel/cosbench/api/openio/OpenIOStorageFactory.java index 610ea2e3..65ae51a0 100644 --- a/dev/cosbench-openio/src/com/intel/cosbench/api/openio/OpenIOStorageFactory.java +++ b/dev/cosbench-openio/src/com/intel/cosbench/api/openio/OpenIOStorageFactory.java @@ -1,17 +1,35 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.openio; import com.intel.cosbench.api.storage.StorageAPI; import com.intel.cosbench.api.storage.StorageAPIFactory; /** - * + * * @author Christopher Dedeurwaerder * */ public class OpenIOStorageFactory implements StorageAPIFactory { private static final String OIO_STORAGE_NAME = "openio"; - + @Override public String getStorageName() { return OIO_STORAGE_NAME; diff --git a/dev/cosbench-openio/src/com/intel/cosbench/client/openio/OioStorageConstants.java b/dev/cosbench-openio/src/com/intel/cosbench/client/openio/OioStorageConstants.java index c353ddab..3171e0c6 100644 --- a/dev/cosbench-openio/src/com/intel/cosbench/client/openio/OioStorageConstants.java +++ b/dev/cosbench-openio/src/com/intel/cosbench/client/openio/OioStorageConstants.java @@ -1,15 +1,34 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.client.openio; /** - * + * * @author Christopher Dedeurwaerder * */ public class OioStorageConstants { public static final String PROXYD_URL_KEY = "proxyd-url"; + public static final String ECD_URL_KEY = "ecd-url"; public static final String ACCOUNT_KEY = "account"; public static final String NS_KEY = "ns"; - + } diff --git a/dev/cosbench-oss/.classpath b/dev/cosbench-oss/.classpath index f0fb6aaf..a4b364b5 100644 --- a/dev/cosbench-oss/.classpath +++ b/dev/cosbench-oss/.classpath @@ -1,14 +1,14 @@ - - - - - - - - - - - - - - + + + + + + + + + + + + + + diff --git a/dev/cosbench-oss/META-INF/MANIFEST.MF b/dev/cosbench-oss/META-INF/MANIFEST.MF index 7daf58be..c3bd45a6 100644 --- a/dev/cosbench-oss/META-INF/MANIFEST.MF +++ b/dev/cosbench-oss/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Cosbench-oss Bundle-SymbolicName: cosbench-oss -Bundle-Version: 0.4.2.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Bundle-ClassPath: ., diff --git a/dev/cosbench-oss/src/com/intel/cosbench/api/oss/OSSStorage.java b/dev/cosbench-oss/src/com/intel/cosbench/api/oss/OSSStorage.java index e81a44c5..55a1b799 100644 --- a/dev/cosbench-oss/src/com/intel/cosbench/api/oss/OSSStorage.java +++ b/dev/cosbench-oss/src/com/intel/cosbench/api/oss/OSSStorage.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.oss; import static com.intel.cosbench.client.oss.OSSConstants.AUTH_PASSWORD_DEFAULT; @@ -34,138 +52,138 @@ public class OSSStorage extends NoneStorage { - private int timeout; - private String accessKey; - private String secretKey; - private String endpoint; - - private OSS ossClient; - - @Override - public void init(Config config, Logger logger) { - super.init(config, logger); - - timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); - - parms.put(CONN_TIMEOUT_KEY, timeout); - - endpoint = config.get(ENDPOINT_KEY, ENDPOINT_DEFAULT); - accessKey = config.get(AUTH_USERNAME_KEY, AUTH_USERNAME_DEFAULT); - secretKey = config.get(AUTH_PASSWORD_KEY, AUTH_PASSWORD_DEFAULT); - - boolean pathStyleAccess = config.getBoolean(PATH_STYLE_ACCESS_KEY, PATH_STYLE_ACCESS_DEFAULT); - - String proxyHost = config.get(PROXY_HOST_KEY, ""); - String proxyPort = config.get(PROXY_PORT_KEY, ""); - - parms.put(ENDPOINT_KEY, endpoint); - parms.put(AUTH_USERNAME_KEY, accessKey); - parms.put(AUTH_PASSWORD_KEY, secretKey); - parms.put(PATH_STYLE_ACCESS_KEY, pathStyleAccess); - parms.put(PROXY_HOST_KEY, proxyHost); - parms.put(PROXY_PORT_KEY, proxyPort); - - logger.debug("using storage config: {}", parms); - - ClientConfiguration clientConf = new ClientConfiguration(); - clientConf.setConnectionTimeout(timeout); - clientConf.setSocketTimeout(timeout); - // clientConf.setProtocol(Protocol.HTTP); - if ((!proxyHost.equals("")) && (!proxyPort.equals(""))) { - clientConf.setProxyHost(proxyHost); - clientConf.setProxyPort(Integer.parseInt(proxyPort)); - } - - Credentials ossCredentials = new DefaultCredentials(accessKey, secretKey); - CredentialsProvider credsProvider = new DefaultCredentialProvider(ossCredentials); - ossClient = new OSSClient(endpoint, credsProvider, clientConf); - - logger.debug("aliyun oss client has been initialized"); - } - - @Override - public void setAuthContext(AuthContext info) { - super.setAuthContext(info); - } - - @Override - public void dispose() { - super.dispose(); - ossClient = null; - } - - @Override - public Context getParms() { - return super.getParms(); - } - - @Override - public InputStream getObject(String container, String object, Config config) { - super.getObject(container, object, config); - InputStream stream; - try { - OSSObject ossObject = ossClient.getObject(container, object); - stream = ossObject.getObjectContent(); - } catch (OSSException ossExce) { - throw new StorageException("error message:" + ossExce.getErrorMessage(), ossExce); - } catch (Exception e) { - throw new StorageException(e); - } - return stream; - } - - @Override - public void createContainer(String container, Config config) { - super.createContainer(container, config); - try { - if (!ossClient.doesBucketExist(container)) { - ossClient.createBucket(container); - } - } catch (OSSException ossExce) { - throw new StorageException(ossExce.getErrorMessage(), ossExce); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void createObject(String container, String object, InputStream data, long length, Config config) { - super.createObject(container, object, data, length, config); - try { - ObjectMetadata metadata = new ObjectMetadata(); - metadata.setContentLength(length); - metadata.setContentType("application/octet-stream"); - ossClient.putObject(container, object, data, metadata); - } catch (OSSException ossExce) { - throw new StorageException(ossExce.getErrorMessage(), ossExce); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void deleteContainer(String container, Config config) { - super.deleteContainer(container, config); - try { - if (ossClient.doesBucketExist(container)) { - ossClient.deleteBucket(container); - } - } catch (OSSException ossExce) { - throw new StorageException(ossExce.getErrorMessage(), ossExce); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void deleteObject(String container, String object, Config config) { - super.deleteObject(container, object, config); - try { - ossClient.deleteObject(container, object); - } catch (OSSException ossExce) { - throw new StorageException(ossExce.getErrorMessage(), ossExce); - } catch (Exception e) { - throw new StorageException(e); - } - } + private int timeout; + private String accessKey; + private String secretKey; + private String endpoint; + + private OSS ossClient; + + @Override + public void init(Config config, Logger logger) { + super.init(config, logger); + + timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); + + parms.put(CONN_TIMEOUT_KEY, timeout); + + endpoint = config.get(ENDPOINT_KEY, ENDPOINT_DEFAULT); + accessKey = config.get(AUTH_USERNAME_KEY, AUTH_USERNAME_DEFAULT); + secretKey = config.get(AUTH_PASSWORD_KEY, AUTH_PASSWORD_DEFAULT); + + boolean pathStyleAccess = config.getBoolean(PATH_STYLE_ACCESS_KEY, PATH_STYLE_ACCESS_DEFAULT); + + String proxyHost = config.get(PROXY_HOST_KEY, ""); + String proxyPort = config.get(PROXY_PORT_KEY, ""); + + parms.put(ENDPOINT_KEY, endpoint); + parms.put(AUTH_USERNAME_KEY, accessKey); + parms.put(AUTH_PASSWORD_KEY, secretKey); + parms.put(PATH_STYLE_ACCESS_KEY, pathStyleAccess); + parms.put(PROXY_HOST_KEY, proxyHost); + parms.put(PROXY_PORT_KEY, proxyPort); + + logger.debug("using storage config: {}", parms); + + ClientConfiguration clientConf = new ClientConfiguration(); + clientConf.setConnectionTimeout(timeout); + clientConf.setSocketTimeout(timeout); + // clientConf.setProtocol(Protocol.HTTP); + if ((!proxyHost.equals("")) && (!proxyPort.equals(""))) { + clientConf.setProxyHost(proxyHost); + clientConf.setProxyPort(Integer.parseInt(proxyPort)); + } + + Credentials ossCredentials = new DefaultCredentials(accessKey, secretKey); + CredentialsProvider credsProvider = new DefaultCredentialProvider(ossCredentials); + ossClient = new OSSClient(endpoint, credsProvider, clientConf); + + logger.debug("aliyun oss client has been initialized"); + } + + @Override + public void setAuthContext(AuthContext info) { + super.setAuthContext(info); + } + + @Override + public void dispose() { + super.dispose(); + ossClient = null; + } + + @Override + public Context getParms() { + return super.getParms(); + } + + @Override + public InputStream getObject(String container, String object, Config config) { + super.getObject(container, object, config); + InputStream stream; + try { + OSSObject ossObject = ossClient.getObject(container, object); + stream = ossObject.getObjectContent(); + } catch (OSSException ossExce) { + throw new StorageException("error message:" + ossExce.getErrorMessage(), ossExce); + } catch (Exception e) { + throw new StorageException(e); + } + return stream; + } + + @Override + public void createContainer(String container, Config config) { + super.createContainer(container, config); + try { + if (!ossClient.doesBucketExist(container)) { + ossClient.createBucket(container); + } + } catch (OSSException ossExce) { + throw new StorageException(ossExce.getErrorMessage(), ossExce); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void createObject(String container, String object, InputStream data, long length, Config config) { + super.createObject(container, object, data, length, config); + try { + ObjectMetadata metadata = new ObjectMetadata(); + metadata.setContentLength(length); + metadata.setContentType("application/octet-stream"); + ossClient.putObject(container, object, data, metadata); + } catch (OSSException ossExce) { + throw new StorageException(ossExce.getErrorMessage(), ossExce); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void deleteContainer(String container, Config config) { + super.deleteContainer(container, config); + try { + if (ossClient.doesBucketExist(container)) { + ossClient.deleteBucket(container); + } + } catch (OSSException ossExce) { + throw new StorageException(ossExce.getErrorMessage(), ossExce); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void deleteObject(String container, String object, Config config) { + super.deleteObject(container, object, config); + try { + ossClient.deleteObject(container, object); + } catch (OSSException ossExce) { + throw new StorageException(ossExce.getErrorMessage(), ossExce); + } catch (Exception e) { + throw new StorageException(e); + } + } } diff --git a/dev/cosbench-oss/src/com/intel/cosbench/api/oss/OSSStorageFactory.java b/dev/cosbench-oss/src/com/intel/cosbench/api/oss/OSSStorageFactory.java index 43831719..dd8d391f 100644 --- a/dev/cosbench-oss/src/com/intel/cosbench/api/oss/OSSStorageFactory.java +++ b/dev/cosbench-oss/src/com/intel/cosbench/api/oss/OSSStorageFactory.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.oss; import com.intel.cosbench.api.storage.StorageAPI; @@ -5,14 +23,14 @@ public class OSSStorageFactory implements StorageAPIFactory { - @Override - public String getStorageName() { - return "oss"; - } + @Override + public String getStorageName() { + return "oss"; + } - @Override - public StorageAPI getStorageAPI() { - return new OSSStorage(); - } + @Override + public StorageAPI getStorageAPI() { + return new OSSStorage(); + } } diff --git a/dev/cosbench-oss/src/com/intel/cosbench/client/oss/OSSConstants.java b/dev/cosbench-oss/src/com/intel/cosbench/client/oss/OSSConstants.java index 943cea8a..33d2e01d 100644 --- a/dev/cosbench-oss/src/com/intel/cosbench/client/oss/OSSConstants.java +++ b/dev/cosbench-oss/src/com/intel/cosbench/client/oss/OSSConstants.java @@ -1,44 +1,62 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.client.oss; public interface OSSConstants { - // -------------------------------------------------------------------------- - // CONNECTION - // -------------------------------------------------------------------------- - - String CONN_TIMEOUT_KEY = "timeout"; - int CONN_TIMEOUT_DEFAULT = 50000; - // -------------------------------------------------------------------------- - // ENDPOINT - // -------------------------------------------------------------------------- - String ENDPOINT_KEY = "endpoint"; - String ENDPOINT_DEFAULT = "oss-cn-hangzhou.aliyuncs.com";// default is 1 - - // -------------------------------------------------------------------------- - // AUTHENTICATION - // -------------------------------------------------------------------------- - - String AUTH_USERNAME_KEY = "accesskey"; - String AUTH_USERNAME_DEFAULT = ""; - - String AUTH_PASSWORD_KEY = "secretkey"; - String AUTH_PASSWORD_DEFAULT = ""; - - // -------------------------------------------------------------------------- - // CLIENT CONFIGURATION - // -------------------------------------------------------------------------- - String PROXY_HOST_KEY = "proxyhost"; - String PROXY_PORT_KEY = "proxyport"; - - // -------------------------------------------------------------------------- - // PATH STYLE ACCESS - // -------------------------------------------------------------------------- - String PATH_STYLE_ACCESS_KEY = "path_style_access"; - boolean PATH_STYLE_ACCESS_DEFAULT = false; - - // -------------------------------------------------------------------------- - // CONTEXT NEEDS FROM AUTH MODULE - // -------------------------------------------------------------------------- - String OSSCLIENT_KEY = "ossclient"; + // -------------------------------------------------------------------------- + // CONNECTION + // -------------------------------------------------------------------------- + + String CONN_TIMEOUT_KEY = "timeout"; + int CONN_TIMEOUT_DEFAULT = 50000; + // -------------------------------------------------------------------------- + // ENDPOINT + // -------------------------------------------------------------------------- + String ENDPOINT_KEY = "endpoint"; + String ENDPOINT_DEFAULT = "oss-cn-hangzhou.aliyuncs.com";// default is 1 + + // -------------------------------------------------------------------------- + // AUTHENTICATION + // -------------------------------------------------------------------------- + + String AUTH_USERNAME_KEY = "accesskey"; + String AUTH_USERNAME_DEFAULT = ""; + + String AUTH_PASSWORD_KEY = "secretkey"; + String AUTH_PASSWORD_DEFAULT = ""; + + // -------------------------------------------------------------------------- + // CLIENT CONFIGURATION + // -------------------------------------------------------------------------- + String PROXY_HOST_KEY = "proxyhost"; + String PROXY_PORT_KEY = "proxyport"; + + // -------------------------------------------------------------------------- + // PATH STYLE ACCESS + // -------------------------------------------------------------------------- + String PATH_STYLE_ACCESS_KEY = "path_style_access"; + boolean PATH_STYLE_ACCESS_DEFAULT = false; + + // -------------------------------------------------------------------------- + // CONTEXT NEEDS FROM AUTH MODULE + // -------------------------------------------------------------------------- + String OSSCLIENT_KEY = "ossclient"; } diff --git a/dev/cosbench-s3/META-INF/MANIFEST.MF b/dev/cosbench-s3/META-INF/MANIFEST.MF index e81b0826..c5c1bdb0 100644 --- a/dev/cosbench-s3/META-INF/MANIFEST.MF +++ b/dev/cosbench-s3/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: S3 Storage Client Bundle Bundle-SymbolicName: cosbench-s3 -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Bundle-ClassPath: ., diff --git a/dev/cosbench-s3/src/com/intel/cosbench/api/S3Stor/S3Storage.java b/dev/cosbench-s3/src/com/intel/cosbench/api/S3Stor/S3Storage.java index 509c56be..b9ec0a1f 100644 --- a/dev/cosbench-s3/src/com/intel/cosbench/api/S3Stor/S3Storage.java +++ b/dev/cosbench-s3/src/com/intel/cosbench/api/S3Stor/S3Storage.java @@ -1,163 +1,225 @@ -package com.intel.cosbench.api.S3Stor; - -import static com.intel.cosbench.client.S3Stor.S3Constants.*; - -import java.io.*; - -import org.apache.http.HttpStatus; - -import com.amazonaws.*; -import com.amazonaws.auth.*; -import com.amazonaws.services.s3.*; -import com.amazonaws.services.s3.model.*; - -import com.intel.cosbench.api.storage.*; -import com.intel.cosbench.api.context.*; -import com.intel.cosbench.config.Config; -import com.intel.cosbench.log.Logger; - -public class S3Storage extends NoneStorage { - private int timeout; - - private String accessKey; - private String secretKey; - private String endpoint; - - private AmazonS3 client; - - @Override - public void init(Config config, Logger logger) { - super.init(config, logger); - - timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); - - parms.put(CONN_TIMEOUT_KEY, timeout); - - endpoint = config.get(ENDPOINT_KEY, ENDPOINT_DEFAULT); - accessKey = config.get(AUTH_USERNAME_KEY, AUTH_USERNAME_DEFAULT); - secretKey = config.get(AUTH_PASSWORD_KEY, AUTH_PASSWORD_DEFAULT); - - boolean pathStyleAccess = config.getBoolean(PATH_STYLE_ACCESS_KEY, PATH_STYLE_ACCESS_DEFAULT); - - String proxyHost = config.get(PROXY_HOST_KEY, ""); - String proxyPort = config.get(PROXY_PORT_KEY, ""); - - parms.put(ENDPOINT_KEY, endpoint); - parms.put(AUTH_USERNAME_KEY, accessKey); - parms.put(AUTH_PASSWORD_KEY, secretKey); - parms.put(PATH_STYLE_ACCESS_KEY, pathStyleAccess); - parms.put(PROXY_HOST_KEY, proxyHost); - parms.put(PROXY_PORT_KEY, proxyPort); - - logger.debug("using storage config: {}", parms); - - ClientConfiguration clientConf = new ClientConfiguration(); - clientConf.setConnectionTimeout(timeout); - clientConf.setSocketTimeout(timeout); - clientConf.withUseExpectContinue(false); - clientConf.withSignerOverride("S3SignerType"); -// clientConf.setProtocol(Protocol.HTTP); - if((!proxyHost.equals(""))&&(!proxyPort.equals(""))){ - clientConf.setProxyHost(proxyHost); - clientConf.setProxyPort(Integer.parseInt(proxyPort)); - } - - AWSCredentials myCredentials = new BasicAWSCredentials(accessKey, secretKey); - client = new AmazonS3Client(myCredentials, clientConf); - client.setEndpoint(endpoint); - client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(pathStyleAccess)); - - logger.debug("S3 client has been initialized"); - } - - @Override - public void setAuthContext(AuthContext info) { - super.setAuthContext(info); -// try { -// client = (AmazonS3)info.get(S3CLIENT_KEY); -// logger.debug("s3client=" + client); -// } catch (Exception e) { -// throw new StorageException(e); -// } - } - - @Override - public void dispose() { - super.dispose(); - client = null; - } - - @Override - public InputStream getObject(String container, String object, Config config) { - super.getObject(container, object, config); - InputStream stream; - try { - - S3Object s3Obj = client.getObject(container, object); - stream = s3Obj.getObjectContent(); - - } catch (Exception e) { - throw new StorageException(e); - } - return stream; - } - - @Override - public void createContainer(String container, Config config) { - super.createContainer(container, config); - try { - if(!client.doesBucketExist(container)) { - - client.createBucket(container); - } - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void createObject(String container, String object, InputStream data, - long length, Config config) { - super.createObject(container, object, data, length, config); - try { - ObjectMetadata metadata = new ObjectMetadata(); - metadata.setContentLength(length); - metadata.setContentType("application/octet-stream"); - - client.putObject(container, object, data, metadata); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void deleteContainer(String container, Config config) { - super.deleteContainer(container, config); - try { - if(client.doesBucketExist(container)) { - client.deleteBucket(container); - } - } catch(AmazonS3Exception awse) { - if(awse.getStatusCode() != HttpStatus.SC_NOT_FOUND) { - throw new StorageException(awse); - } - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void deleteObject(String container, String object, Config config) { - super.deleteObject(container, object, config); - try { - client.deleteObject(container, object); - } catch(AmazonS3Exception awse) { - if(awse.getStatusCode() != HttpStatus.SC_NOT_FOUND) { - throw new StorageException(awse); - } - } catch (Exception e) { - throw new StorageException(e); - } - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ +package com.intel.cosbench.api.S3Stor; + +import static com.intel.cosbench.client.S3Stor.S3Constants.*; + +import java.io.*; + +import org.apache.http.HttpStatus; + +import com.amazonaws.*; +import com.amazonaws.auth.*; +import com.amazonaws.services.s3.*; +import com.amazonaws.services.s3.model.*; + +import com.intel.cosbench.api.storage.*; +import com.intel.cosbench.api.context.*; +import com.intel.cosbench.config.Config; +import com.intel.cosbench.log.Logger; + +public class S3Storage extends NoneStorage { + private int timeout; + + private String accessKey; + private String secretKey; + private String endpoint; + + private AmazonS3 client; + + @Override + public void init(Config config, Logger logger) { + super.init(config, logger); + + timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); + + parms.put(CONN_TIMEOUT_KEY, timeout); + + endpoint = config.get(ENDPOINT_KEY, ENDPOINT_DEFAULT); + accessKey = config.get(AUTH_USERNAME_KEY, AUTH_USERNAME_DEFAULT); + secretKey = config.get(AUTH_PASSWORD_KEY, AUTH_PASSWORD_DEFAULT); + + boolean pathStyleAccess = config.getBoolean(PATH_STYLE_ACCESS_KEY, PATH_STYLE_ACCESS_DEFAULT); + int maxConnections = config.getInt(MAX_CONNECTIONS, MAX_CONNECTIONS_DEFAULT); + + String proxyHost = config.get(PROXY_HOST_KEY, ""); + String proxyPort = config.get(PROXY_PORT_KEY, ""); + + parms.put(ENDPOINT_KEY, endpoint); + parms.put(AUTH_USERNAME_KEY, accessKey); + parms.put(AUTH_PASSWORD_KEY, secretKey); + parms.put(PATH_STYLE_ACCESS_KEY, pathStyleAccess); + parms.put(MAX_CONNECTIONS, maxConnections); + parms.put(PROXY_HOST_KEY, proxyHost); + parms.put(PROXY_PORT_KEY, proxyPort); + + initClient(); + } + + private AmazonS3 initClient() { + logger.debug("initialize S3 client with storage config: {}", parms); + + ClientConfiguration clientConf = new ClientConfiguration(); + clientConf.setConnectionTimeout(parms.getInt(CONN_TIMEOUT_KEY)); + clientConf.setMaxConnections(parms.getInt(MAX_CONNECTIONS)); + clientConf.setSocketTimeout(timeout); + clientConf.withUseExpectContinue(false); + clientConf.withSignerOverride("S3SignerType"); + System.setProperty(SDKGlobalConfiguration.ENABLE_S3_SIGV4_SYSTEM_PROPERTY, "true"); + if((!parms.getStr(PROXY_HOST_KEY).equals(""))&&(!parms.getStr(PROXY_PORT_KEY).equals(""))){ + clientConf.setProxyHost(parms.getStr(PROXY_HOST_KEY)); + clientConf.setProxyPort(parms.getInt(PROXY_PORT_KEY)); + } + + AWSCredentials myCredentials = new BasicAWSCredentials(accessKey, secretKey); + client = new AmazonS3Client(myCredentials, clientConf); + client.setEndpoint(endpoint); + client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(parms.getBoolean(PATH_STYLE_ACCESS_KEY))); + + logger.debug("S3 client has been initialized"); + + return client; + } + + @Override + public void setAuthContext(AuthContext info) { + super.setAuthContext(info); + } + + @Override + public void dispose() { + super.dispose(); + client = null; + } + + @Override + public InputStream getObject(String container, String object, Config config) { + super.getObject(container, object, config); + InputStream stream = null; + try { + + S3Object s3Obj = client.getObject(container, object); + stream = s3Obj.getObjectContent(); + + } catch(AmazonServiceException ase) { + if(ase.getStatusCode() != HttpStatus.SC_NOT_FOUND) { + throw new StorageException(ase); + } + } catch (AmazonClientException ace) { // recreate the AmazonS3 client connection if it is broken. + logger.warn("below exception encountered when retrieving object " + object + " at bucket " + container + ": " + ace.getMessage()); + ace.printStackTrace(); + initClient(); + } + + return stream; + } + + @Override + public void createContainer(String container, Config config) { + super.createContainer(container, config); + try { + container = container.split("/")[0]; + if(!client.doesBucketExist(container)) { + client.createBucket(container); + } + } catch(AmazonServiceException ase) { + if(ase.getStatusCode() != HttpStatus.SC_NOT_FOUND) { + throw new StorageException(ase); + } + } catch (AmazonClientException ace) { // recreate the AmazonS3 client connection if it is broken. + logger.warn("below exception encountered when creating bucket " + container + ": " + ace.getMessage()); + ace.printStackTrace(); + initClient(); + } + } + + @Override + public void createObject(String container, String object, InputStream data, + long length, Config config) { + super.createObject(container, object, data, length, config); + try { + ObjectMetadata metadata = new ObjectMetadata(); + metadata.setContentLength(length); + metadata.setContentType("application/octet-stream"); + + client.putObject(container, object, data, metadata); + }catch(AmazonServiceException ase) { + if(ase.getStatusCode() != HttpStatus.SC_NOT_FOUND) { + throw new StorageException(ase); + } + } catch (AmazonClientException ace) { // recreate the AmazonS3 client connection if it is broken. + logger.warn("below exception encountered when creating object " + object + " at " + container + ": " + ace.getMessage()); + ace.printStackTrace(); + initClient(); + } + } + + @Override + public void deleteContainer(String container, Config config) { + super.deleteContainer(container, config); + try { + container = container.split("/")[0]; + if(client.doesBucketExist(container)) { + client.deleteBucket(container); + } + } catch(AmazonServiceException ase) { + if(ase.getStatusCode() != HttpStatus.SC_NOT_FOUND) { + throw new StorageException(ase); + } + } catch (AmazonClientException ace) { // recreate the AmazonS3 client connection if it is broken. + logger.warn("below exception encountered when deleting bucket " + container + ": " + ace.getMessage()); + ace.printStackTrace(); + initClient(); + } + } + + @Override + public void deleteObject(String container, String object, Config config) { + super.deleteObject(container, object, config); + try { + client.deleteObject(container, object); + } catch(AmazonServiceException ase) { + if(ase.getStatusCode() != HttpStatus.SC_NOT_FOUND) { + throw new StorageException(ase); + } + } catch (AmazonClientException ace) { // recreate the AmazonS3 client connection if it is broken. + logger.warn("below exception encountered when deleting object " + object + " at bucket " + container + ": " + ace.getMessage()); + ace.printStackTrace(); + initClient(); + } + } + + @Override + public InputStream getList(String container, String object, Config config) { + super.getList(container, object, config); + InputStream stream = null; + try { + ObjectListing listing = client.listObjects(container, object); + stream = new ByteArrayInputStream(listing.getObjectSummaries().toString().getBytes()); + } catch(AmazonServiceException ase) { + if(ase.getStatusCode() != HttpStatus.SC_NOT_FOUND) { + throw new StorageException(ase); + } + } catch (AmazonClientException ace) { // recreate the AmazonS3 client connection if it is broken. + logger.warn("below exception encountered when listing object " + object + " at bucket " + container + ": " + ace.getMessage()); + ace.printStackTrace(); + initClient(); + } + return stream; + } +} diff --git a/dev/cosbench-s3/src/com/intel/cosbench/api/S3Stor/S3StorageFactory.java b/dev/cosbench-s3/src/com/intel/cosbench/api/S3Stor/S3StorageFactory.java index eb51a6d9..688dc263 100644 --- a/dev/cosbench-s3/src/com/intel/cosbench/api/S3Stor/S3StorageFactory.java +++ b/dev/cosbench-s3/src/com/intel/cosbench/api/S3Stor/S3StorageFactory.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.api.S3Stor; import com.intel.cosbench.api.storage.*; diff --git a/dev/cosbench-s3/src/com/intel/cosbench/client/S3Stor/S3Constants.java b/dev/cosbench-s3/src/com/intel/cosbench/client/S3Stor/S3Constants.java index 7393af62..81c65309 100644 --- a/dev/cosbench-s3/src/com/intel/cosbench/client/S3Stor/S3Constants.java +++ b/dev/cosbench-s3/src/com/intel/cosbench/client/S3Stor/S3Constants.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.client.S3Stor; public interface S3Constants { @@ -19,16 +37,21 @@ public interface S3Constants { String AUTH_USERNAME_KEY = "accesskey"; String AUTH_USERNAME_DEFAULT = ""; - + String AUTH_PASSWORD_KEY = "secretkey"; String AUTH_PASSWORD_DEFAULT = ""; - + // -------------------------------------------------------------------------- // CLIENT CONFIGURATION // -------------------------------------------------------------------------- String PROXY_HOST_KEY = "proxyhost"; - String PROXY_PORT_KEY = "proxyport"; - + String PROXY_PORT_KEY = "proxyport"; + + // MAX CONNECTIONS DEFAULT + // -------------------------------------------------------------------------- + String MAX_CONNECTIONS = "max_connections"; + int MAX_CONNECTIONS_DEFAULT = 50; + // -------------------------------------------------------------------------- // PATH STYLE ACCESS // -------------------------------------------------------------------------- diff --git a/dev/cosbench-scality/META-INF/MANIFEST.MF b/dev/cosbench-scality/META-INF/MANIFEST.MF index 1fadd176..fe3b1ef5 100644 --- a/dev/cosbench-scality/META-INF/MANIFEST.MF +++ b/dev/cosbench-scality/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Scality COSBench Client Bundle Bundle-SymbolicName: cosbench-scality -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: Scality Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-scality/src/com/scality/cosbench/api/SproxydStorage.java b/dev/cosbench-scality/src/com/scality/cosbench/api/SproxydStorage.java index 0c290adb..dde7518e 100644 --- a/dev/cosbench-scality/src/com/scality/cosbench/api/SproxydStorage.java +++ b/dev/cosbench-scality/src/com/scality/cosbench/api/SproxydStorage.java @@ -1,5 +1,23 @@ /** +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -52,47 +70,47 @@ public class SproxydStorage extends NoneStorage { - private static HttpClient httpClient; - private static int nbClients = 0; - private SproxydClient client; - private String basePath; - private String hosts; - private int port; - private int timeout; - private int maxTotal; - private int maxPerRoute; - private boolean logging; - - @Override - public void init(Config config, Logger logger) { - super.init(config, logger); - initParms(config); - - - HttpClient httpClient = createHttpClient(timeout, maxTotal, maxPerRoute); - client = new SproxydClient(httpClient, hosts, port, basePath, logging ? logger : null); - logger.debug("sproxyd client has been initialized"); - } - - private static synchronized HttpClient createHttpClient(int timeout, int maxTotal, int maxPerRoute) { - if (httpClient == null) { - final ThreadSafeClientConnManager tscm = new ThreadSafeClientConnManager(); - tscm.setMaxTotal(maxTotal); - tscm.setDefaultMaxPerRoute(maxPerRoute); - final HttpParams params = createDefaultHttpParams(timeout); - httpClient = new DefaultHttpClient(tscm, params); - } - nbClients++; - return httpClient; - } - - private static synchronized void disposeHttpClient() { - nbClients--; - if (nbClients == 0) { - httpClient.getConnectionManager().shutdown(); - httpClient = null; - } - } + private static HttpClient httpClient; + private static int nbClients = 0; + private SproxydClient client; + private String basePath; + private String hosts; + private int port; + private int timeout; + private int maxTotal; + private int maxPerRoute; + private boolean logging; + + @Override + public void init(Config config, Logger logger) { + super.init(config, logger); + initParms(config); + + + HttpClient httpClient = createHttpClient(timeout, maxTotal, maxPerRoute); + client = new SproxydClient(httpClient, hosts, port, basePath, logging ? logger : null); + logger.debug("sproxyd client has been initialized"); + } + + private static synchronized HttpClient createHttpClient(int timeout, int maxTotal, int maxPerRoute) { + if (httpClient == null) { + final ThreadSafeClientConnManager tscm = new ThreadSafeClientConnManager(); + tscm.setMaxTotal(maxTotal); + tscm.setDefaultMaxPerRoute(maxPerRoute); + final HttpParams params = createDefaultHttpParams(timeout); + httpClient = new DefaultHttpClient(tscm, params); + } + nbClients++; + return httpClient; + } + + private static synchronized void disposeHttpClient() { + nbClients--; + if (nbClients == 0) { + httpClient.getConnectionManager().shutdown(); + httpClient = null; + } + } private static HttpParams createDefaultHttpParams(int timeout) { HttpParams params = new BasicHttpParams(); @@ -105,126 +123,126 @@ private static HttpParams createDefaultHttpParams(int timeout) { HttpProtocolParams.setUserAgent(params, "cosbench/2.0"); return params; } - private void initParms(Config config) { - basePath = config.get(BASE_PATH_KEY, BASE_PATH_DEFAULT); - hosts = config.get(HOSTS_KEY, HOSTS_DEFAULT); - port = config.getInt(PORT_KEY, PORT_DEFAULT); - timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); - logging = config.getBoolean(LOGGING_KEY, LOGGING_DEFAULT); - final String poolSize[] = config.get(POOL_SIZE_KEY, POOL_SIZE_DEFAULT).split(","); - try { - maxTotal = Integer.parseInt(poolSize[0]); - maxPerRoute = poolSize.length > 1 ? Integer.parseInt(poolSize[1]) : 2; - } catch (NumberFormatException exc) { - throw new ConfigException("Invalid pool size, must be , or ", exc); - } - /* register all parameters */ - parms.put(LOGGING_KEY, logging); - - parms.put(CONN_TIMEOUT_KEY, timeout); - - logger.debug("using storage config: {}", parms); - } - - @Override - public void dispose() { - super.dispose(); - client.dispose(); - disposeHttpClient(); - } - - @Override - public void abort() { - super.abort(); - client.abort(); - } - - @Override - public InputStream getObject(String container, String object, Config config) { - super.getObject(container, object, config); - InputStream stream; - try { - stream = client.getObjectAsStream(container, object); - } catch (SocketTimeoutException te) { - throw new StorageTimeoutException(te); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SproxydClientException se) { - String msg = se.getMessage();//se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - return stream; - } - - @Override - public void createContainer(String container, Config config) { - super.createContainer(container, config); - try { - client.createContainer(container); - } catch (SocketTimeoutException te) { - throw new StorageTimeoutException(te); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SproxydClientException se) { - String msg = se.getMessage();//se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void createObject(String container, String object, InputStream data, - long length, Config config) { - super.createObject(container, object, data, length, config); - try { - client.storeStreamedObject(container, object, data, length); - } catch (SocketTimeoutException te) { - throw new StorageTimeoutException(te); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SproxydClientException se) { - String msg = se.getMessage();//se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void deleteContainer(String container, Config config) { - super.deleteContainer(container, config); - try { - client.deleteContainer(container); - } catch (SocketTimeoutException te) { - throw new StorageTimeoutException(te); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SproxydClientException se) { - String msg = se.getMessage();//se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void deleteObject(String container, String object, Config config) { - super.deleteObject(container, object, config); - try { - client.deleteObject(container, object); - } catch (SocketTimeoutException te) { - throw new StorageTimeoutException(te); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SproxydClientException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } + private void initParms(Config config) { + basePath = config.get(BASE_PATH_KEY, BASE_PATH_DEFAULT); + hosts = config.get(HOSTS_KEY, HOSTS_DEFAULT); + port = config.getInt(PORT_KEY, PORT_DEFAULT); + timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); + logging = config.getBoolean(LOGGING_KEY, LOGGING_DEFAULT); + final String poolSize[] = config.get(POOL_SIZE_KEY, POOL_SIZE_DEFAULT).split(","); + try { + maxTotal = Integer.parseInt(poolSize[0]); + maxPerRoute = poolSize.length > 1 ? Integer.parseInt(poolSize[1]) : 2; + } catch (NumberFormatException exc) { + throw new ConfigException("Invalid pool size, must be , or ", exc); + } + /* register all parameters */ + parms.put(LOGGING_KEY, logging); + + parms.put(CONN_TIMEOUT_KEY, timeout); + + logger.debug("using storage config: {}", parms); + } + + @Override + public void dispose() { + super.dispose(); + client.dispose(); + disposeHttpClient(); + } + + @Override + public void abort() { + super.abort(); + client.abort(); + } + + @Override + public InputStream getObject(String container, String object, Config config) { + super.getObject(container, object, config); + InputStream stream; + try { + stream = client.getObjectAsStream(container, object); + } catch (SocketTimeoutException te) { + throw new StorageTimeoutException(te); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SproxydClientException se) { + String msg = se.getMessage();//se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + return stream; + } + + @Override + public void createContainer(String container, Config config) { + super.createContainer(container, config); + try { + client.createContainer(container); + } catch (SocketTimeoutException te) { + throw new StorageTimeoutException(te); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SproxydClientException se) { + String msg = se.getMessage();//se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void createObject(String container, String object, InputStream data, + long length, Config config) { + super.createObject(container, object, data, length, config); + try { + client.storeStreamedObject(container, object, data, length); + } catch (SocketTimeoutException te) { + throw new StorageTimeoutException(te); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SproxydClientException se) { + String msg = se.getMessage();//se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void deleteContainer(String container, Config config) { + super.deleteContainer(container, config); + try { + client.deleteContainer(container); + } catch (SocketTimeoutException te) { + throw new StorageTimeoutException(te); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SproxydClientException se) { + String msg = se.getMessage();//se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void deleteObject(String container, String object, Config config) { + super.deleteObject(container, object, config); + try { + client.deleteObject(container, object); + } catch (SocketTimeoutException te) { + throw new StorageTimeoutException(te); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SproxydClientException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } } diff --git a/dev/cosbench-scality/src/com/scality/cosbench/api/SproxydStorageFactory.java b/dev/cosbench-scality/src/com/scality/cosbench/api/SproxydStorageFactory.java index 34963121..5573c7e1 100644 --- a/dev/cosbench-scality/src/com/scality/cosbench/api/SproxydStorageFactory.java +++ b/dev/cosbench-scality/src/com/scality/cosbench/api/SproxydStorageFactory.java @@ -1,5 +1,23 @@ /** +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at diff --git a/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydClient.java b/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydClient.java index a7bd9af3..950f86dd 100644 --- a/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydClient.java +++ b/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydClient.java @@ -1,5 +1,23 @@ /** +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -40,375 +58,375 @@ public class SproxydClient { - public static final int DEFAULT_SO_TIMEOUT = 2000; - private static final int DEFAULT_CHECK_PERIOD = 10000; - private static Logger LOG = null; - private HttpClient client; - private SproxydUrlList urls; - private final Map endpoints = new HashMap(); - /* current operation */ - private volatile HttpUriRequest request; - - public SproxydClient(HttpClient client, String hosts, int port, String basePath, Logger logger) { - this.client = client; - urls = new SproxydUrlList(hosts, port, basePath); - urls.registerEndpoints(endpoints); - LOG = logger; - } - - public void dispose() { - request = null; - } - - public void abort() { - if (request != null) - request.abort(); - request = null; - } - - public void createContainer(String container) throws IOException, - SproxydClientException { - // sproxyd by path, container will used as path prefix but does not actually exist - } - - public void deleteContainer(String container) throws IOException, - SproxydClientException { - // see createContainer - } - - public InputStream getObjectAsStream(String container, String object) - throws IOException, SproxydClientException { - - HttpResponse response = null; - int code = HttpStatus.SC_OK; - final SproxydUrl base = urls.getNext(); - try - { - final String url = String.format("%s%s/%s", base, container, object); - final HttpGet get = HttpClientUtil.makeHttpGet(url); - request = get; - response = client.execute(get); - - final StatusLine statusLine = response.getStatusLine(); - code = statusLine.getStatusCode(); - if (code == HttpStatus.SC_OK) { - final InputStream content = response.getEntity().getContent(); - response = null; - return content; - } else if (code == HttpStatus.SC_NOT_FOUND) { - final InputStream content = response.getEntity().getContent(); - response = null; - return content; - } - } catch (IOException exc) { - base.exclude(exc.getMessage()); - throw exc; - } finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - if (code == HttpStatus.SC_INTERNAL_SERVER_ERROR) { - //base.exclude(statusLine.getReasonPhrase()); - } - throw new SproxydClientException(code, "failed to read object"); - } - - public void storeStreamedObject(String container, String object, - InputStream data, long length) throws IOException, SproxydClientException { - HttpResponse response = null; - int code = HttpStatus.SC_OK; - final SproxydUrl base = urls.getNext(); - int trials = 3; - while (trials > 0) { - try { - final String url = String.format("%s%s/%s", base, container, object); - final HttpPut put = HttpClientUtil.makeHttpPut(url); - request = put; - final InputStreamEntity entity = new InputStreamEntity(data, length); - entity.setContentType("application/octet-stream"); - if (length < 0) { - entity.setChunked(true); - } - put.setEntity(entity); - response = client.execute(put); - final StatusLine statusLine = response.getStatusLine(); - code = statusLine.getStatusCode(); - } catch (IOException exc) { - base.exclude(exc.getMessage()); - throw exc; - } finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - // retry locked and internal server errors when allowed to - if ((code != HttpStatus.SC_INTERNAL_SERVER_ERROR) && (code != HttpStatus.SC_LOCKED)) { - break; - } - final Header[] retryAllowed = response.getHeaders("X-Scal-Retry-Allowed"); - if ((code == HttpStatus.SC_INTERNAL_SERVER_ERROR) && (retryAllowed.length > 0) && retryAllowed[0].equals("No")) { - break; - } - if (code == HttpStatus.SC_LOCKED) { - try { - Thread.sleep(500); - } catch (InterruptedException e) { - throw new SproxydClientException(code, e.getLocalizedMessage()); - } - } - trials--; - } - if ((code != HttpStatus.SC_CREATED) && (code != HttpStatus.SC_OK)) { - throw new SproxydClientException(code, "failed to put object"); - } - } - - public void deleteObject(String container, String object) - throws IOException, SproxydClientException { - HttpResponse response = null; - int code = HttpStatus.SC_OK; - final SproxydUrl base = urls.getNext(); - try { - final String url = String.format("%s%s/%s", base, container, object); - final HttpDelete delete = HttpClientUtil.makeHttpDelete(url); - request = delete; - response = client.execute(delete); - final StatusLine statusLine = response.getStatusLine(); - code = statusLine.getStatusCode(); - } catch (IOException exc) { - base.exclude(exc.getMessage()); - throw exc; - } finally { - if (response != null) - EntityUtils.consume(response.getEntity()); - } - if (code == HttpStatus.SC_INTERNAL_SERVER_ERROR) { - //base.exclude(statusLine.getReasonPhrase()); - } - if (code != HttpStatus.SC_OK) { - throw new SproxydClientException(code, "failed to delete object"); - } - } - - /** - * Key of an sproxyd server (host + port) for blacklisting purposes. - * - */ - private static class Endpoint { - private final String host; - private final int port; - - Endpoint(URI uri) { - this.host= uri.getHost(); - this.port = uri.getPort(); - } - - public String getHost() { - return host; - } - - public int getPort() { - return port; - } - - @Override - public int hashCode() { - return getHost().hashCode() + getPort(); - } - - @Override - public boolean equals(Object obj) { - if (obj instanceof Endpoint) { - final Endpoint other = (Endpoint)obj; - return (getPort() == other.getPort()) && getHost().equals(other.getHost()); - } - return false; - } - - @Override - public String toString() { - return getHost()+":"+getPort(); - } - } - - /** - * Background task that monitors an endpoint for blacklisting. - * - */ - private static class SproxydEndpoint extends Thread { - private static int id = 0; - private static int checkPeriod = DEFAULT_CHECK_PERIOD; - private final URI uri; - private HttpClient client; - private boolean isUp; - private volatile boolean shutdownRequested = false; - - SproxydEndpoint(URI uri) { - this.uri = uri; - this.client = HttpClientUtil.createHttpClient(300); - this.setName(String.format("ScalConnChecker-%d", id++)); - this.setDaemon(true); - isUp = true; - } - - public synchronized boolean isUp() { - return isUp; - } - - public synchronized boolean setUp(boolean isUp) { - final boolean before = this.isUp; - this.isUp = isUp; - return before; - } - - private void update(boolean isUp, String message) { - boolean wasUp = setUp(isUp); - if (wasUp && !isUp) { - warn("Blacklisting {}, {}", uri, message); - - } else if (!wasUp) { - if (isUp) { - warn("{} is up", uri); - } else { - warn("{} still down", uri); - } - } - } - - public void exclude(String message) { - update(false, message); - } - - public void include() { - update(true, ""); - } - - public void retryConnection() { - if (isUp()) { - //LOG.debug("Checking connection to {}", uri.toString()); - return; - } else { - info("Checking connection to {}", uri.toString()); - } - HttpResponse response = null; - final HttpGet get = HttpClientUtil.makeHttpGet(uri.toString() + ".conf"); - try { - response = client.execute(get); - final StatusLine statusLine = response.getStatusLine(); - final int statusCode = statusLine.getStatusCode(); - if (statusCode == HttpStatus.SC_OK) { - include(); - } else if (statusCode >= HttpStatus.SC_INTERNAL_SERVER_ERROR) { - exclude(statusLine.getReasonPhrase()); - } - } catch (Exception e) { - exclude(e.getMessage()); - } - } - - @SuppressWarnings("unused") - public void shutdown() { - shutdownRequested = true; - this.interrupt(); - } - - @Override - public void run() { - info("{} thread starting", getName()); - while (!shutdownRequested) { - retryConnection(); - try { - Thread.sleep(checkPeriod); - } catch (InterruptedException e) { - } - } - info("{} thread exiting", getName()); - } - } - - static class SproxydUrl { - private final URI uri; - private SproxydEndpoint endpoint; - - SproxydUrl(String url) throws URISyntaxException { - if (!url.endsWith("/")) { - url += "/"; - } - uri = new URI(url); - } - - public boolean isUp() { - return endpoint.isUp(); - } - - public void exclude(String message) { - endpoint.exclude(message); - } - - public void registerEndpoint(final Map endpoints) { - final Endpoint key = new Endpoint(uri); - if (endpoints.containsKey(key)) { - endpoint = endpoints.get(key); - } else { - endpoint = new SproxydEndpoint(uri); - endpoints.put(key, endpoint); - } - } - - @Override - public String toString() { - return uri.toString(); - } - } - - private static class SproxydUrlList { - private SproxydUrl urls[]; - private int current = 0; - - SproxydUrlList(String hosts, int port, String basePath) { - final String[] hostList = hosts.split(","); - urls = new SproxydUrl[hostList.length]; - for (int i=0; i< hostList.length; i++) { - try { - urls[i] = new SproxydUrl(String.format("http://%s:%d%s", hostList[i], port, basePath)); - } catch (URISyntaxException e) { - } - } - } - - void registerEndpoints(final Map endpoints) { - for (SproxydUrl url : urls) { - url.registerEndpoint(endpoints); - } - } - - synchronized SproxydUrl getNext() throws SproxydClientException { - int trials = urls.length; - while (trials > 0) { - current = (current + 1) % urls.length; - final SproxydUrl url = urls[current]; - if (url.isUp()) return url; - trials -= 1; - } - throw new SproxydClientException(0, "All servers are blacklisted"); - } - - @Override - public String toString() { - return Arrays.toString(urls); - } - - } - - private static void info(String format, Object arg) { - if (LOG != null) LOG.info(format, arg); - } - - private static void warn(String format, Object arg) { - if (LOG != null) LOG.warn(format, arg); - } - - private static void warn(String format, Object arg1, Object arg2) { - if (LOG != null) LOG.warn(format, arg1, arg2); - } + public static final int DEFAULT_SO_TIMEOUT = 2000; + private static final int DEFAULT_CHECK_PERIOD = 10000; + private static Logger LOG = null; + private HttpClient client; + private SproxydUrlList urls; + private final Map endpoints = new HashMap(); + /* current operation */ + private volatile HttpUriRequest request; + + public SproxydClient(HttpClient client, String hosts, int port, String basePath, Logger logger) { + this.client = client; + urls = new SproxydUrlList(hosts, port, basePath); + urls.registerEndpoints(endpoints); + LOG = logger; + } + + public void dispose() { + request = null; + } + + public void abort() { + if (request != null) + request.abort(); + request = null; + } + + public void createContainer(String container) throws IOException, + SproxydClientException { + // sproxyd by path, container will used as path prefix but does not actually exist + } + + public void deleteContainer(String container) throws IOException, + SproxydClientException { + // see createContainer + } + + public InputStream getObjectAsStream(String container, String object) + throws IOException, SproxydClientException { + + HttpResponse response = null; + int code = HttpStatus.SC_OK; + final SproxydUrl base = urls.getNext(); + try + { + final String url = String.format("%s%s/%s", base, container, object); + final HttpGet get = HttpClientUtil.makeHttpGet(url); + request = get; + response = client.execute(get); + + final StatusLine statusLine = response.getStatusLine(); + code = statusLine.getStatusCode(); + if (code == HttpStatus.SC_OK) { + final InputStream content = response.getEntity().getContent(); + response = null; + return content; + } else if (code == HttpStatus.SC_NOT_FOUND) { + final InputStream content = response.getEntity().getContent(); + response = null; + return content; + } + } catch (IOException exc) { + base.exclude(exc.getMessage()); + throw exc; + } finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + if (code == HttpStatus.SC_INTERNAL_SERVER_ERROR) { + //base.exclude(statusLine.getReasonPhrase()); + } + throw new SproxydClientException(code, "failed to read object"); + } + + public void storeStreamedObject(String container, String object, + InputStream data, long length) throws IOException, SproxydClientException { + HttpResponse response = null; + int code = HttpStatus.SC_OK; + final SproxydUrl base = urls.getNext(); + int trials = 3; + while (trials > 0) { + try { + final String url = String.format("%s%s/%s", base, container, object); + final HttpPut put = HttpClientUtil.makeHttpPut(url); + request = put; + final InputStreamEntity entity = new InputStreamEntity(data, length); + entity.setContentType("application/octet-stream"); + if (length < 0) { + entity.setChunked(true); + } + put.setEntity(entity); + response = client.execute(put); + final StatusLine statusLine = response.getStatusLine(); + code = statusLine.getStatusCode(); + } catch (IOException exc) { + base.exclude(exc.getMessage()); + throw exc; + } finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + // retry locked and internal server errors when allowed to + if ((code != HttpStatus.SC_INTERNAL_SERVER_ERROR) && (code != HttpStatus.SC_LOCKED)) { + break; + } + final Header[] retryAllowed = response.getHeaders("X-Scal-Retry-Allowed"); + if ((code == HttpStatus.SC_INTERNAL_SERVER_ERROR) && (retryAllowed.length > 0) && retryAllowed[0].equals("No")) { + break; + } + if (code == HttpStatus.SC_LOCKED) { + try { + Thread.sleep(500); + } catch (InterruptedException e) { + throw new SproxydClientException(code, e.getLocalizedMessage()); + } + } + trials--; + } + if ((code != HttpStatus.SC_CREATED) && (code != HttpStatus.SC_OK)) { + throw new SproxydClientException(code, "failed to put object"); + } + } + + public void deleteObject(String container, String object) + throws IOException, SproxydClientException { + HttpResponse response = null; + int code = HttpStatus.SC_OK; + final SproxydUrl base = urls.getNext(); + try { + final String url = String.format("%s%s/%s", base, container, object); + final HttpDelete delete = HttpClientUtil.makeHttpDelete(url); + request = delete; + response = client.execute(delete); + final StatusLine statusLine = response.getStatusLine(); + code = statusLine.getStatusCode(); + } catch (IOException exc) { + base.exclude(exc.getMessage()); + throw exc; + } finally { + if (response != null) + EntityUtils.consume(response.getEntity()); + } + if (code == HttpStatus.SC_INTERNAL_SERVER_ERROR) { + //base.exclude(statusLine.getReasonPhrase()); + } + if (code != HttpStatus.SC_OK) { + throw new SproxydClientException(code, "failed to delete object"); + } + } + + /** + * Key of an sproxyd server (host + port) for blacklisting purposes. + * + */ + private static class Endpoint { + private final String host; + private final int port; + + Endpoint(URI uri) { + this.host= uri.getHost(); + this.port = uri.getPort(); + } + + public String getHost() { + return host; + } + + public int getPort() { + return port; + } + + @Override + public int hashCode() { + return getHost().hashCode() + getPort(); + } + + @Override + public boolean equals(Object obj) { + if (obj instanceof Endpoint) { + final Endpoint other = (Endpoint)obj; + return (getPort() == other.getPort()) && getHost().equals(other.getHost()); + } + return false; + } + + @Override + public String toString() { + return getHost()+":"+getPort(); + } + } + + /** + * Background task that monitors an endpoint for blacklisting. + * + */ + private static class SproxydEndpoint extends Thread { + private static int id = 0; + private static int checkPeriod = DEFAULT_CHECK_PERIOD; + private final URI uri; + private HttpClient client; + private boolean isUp; + private volatile boolean shutdownRequested = false; + + SproxydEndpoint(URI uri) { + this.uri = uri; + this.client = HttpClientUtil.createHttpClient(300); + this.setName(String.format("ScalConnChecker-%d", id++)); + this.setDaemon(true); + isUp = true; + } + + public synchronized boolean isUp() { + return isUp; + } + + public synchronized boolean setUp(boolean isUp) { + final boolean before = this.isUp; + this.isUp = isUp; + return before; + } + + private void update(boolean isUp, String message) { + boolean wasUp = setUp(isUp); + if (wasUp && !isUp) { + warn("Blacklisting {}, {}", uri, message); + + } else if (!wasUp) { + if (isUp) { + warn("{} is up", uri); + } else { + warn("{} still down", uri); + } + } + } + + public void exclude(String message) { + update(false, message); + } + + public void include() { + update(true, ""); + } + + public void retryConnection() { + if (isUp()) { + //LOG.debug("Checking connection to {}", uri.toString()); + return; + } else { + info("Checking connection to {}", uri.toString()); + } + HttpResponse response = null; + final HttpGet get = HttpClientUtil.makeHttpGet(uri.toString() + ".conf"); + try { + response = client.execute(get); + final StatusLine statusLine = response.getStatusLine(); + final int statusCode = statusLine.getStatusCode(); + if (statusCode == HttpStatus.SC_OK) { + include(); + } else if (statusCode >= HttpStatus.SC_INTERNAL_SERVER_ERROR) { + exclude(statusLine.getReasonPhrase()); + } + } catch (Exception e) { + exclude(e.getMessage()); + } + } + + @SuppressWarnings("unused") + public void shutdown() { + shutdownRequested = true; + this.interrupt(); + } + + @Override + public void run() { + info("{} thread starting", getName()); + while (!shutdownRequested) { + retryConnection(); + try { + Thread.sleep(checkPeriod); + } catch (InterruptedException e) { + } + } + info("{} thread exiting", getName()); + } + } + + static class SproxydUrl { + private final URI uri; + private SproxydEndpoint endpoint; + + SproxydUrl(String url) throws URISyntaxException { + if (!url.endsWith("/")) { + url += "/"; + } + uri = new URI(url); + } + + public boolean isUp() { + return endpoint.isUp(); + } + + public void exclude(String message) { + endpoint.exclude(message); + } + + public void registerEndpoint(final Map endpoints) { + final Endpoint key = new Endpoint(uri); + if (endpoints.containsKey(key)) { + endpoint = endpoints.get(key); + } else { + endpoint = new SproxydEndpoint(uri); + endpoints.put(key, endpoint); + } + } + + @Override + public String toString() { + return uri.toString(); + } + } + + private static class SproxydUrlList { + private SproxydUrl urls[]; + private int current = 0; + + SproxydUrlList(String hosts, int port, String basePath) { + final String[] hostList = hosts.split(","); + urls = new SproxydUrl[hostList.length]; + for (int i=0; i< hostList.length; i++) { + try { + urls[i] = new SproxydUrl(String.format("http://%s:%d%s", hostList[i], port, basePath)); + } catch (URISyntaxException e) { + } + } + } + + void registerEndpoints(final Map endpoints) { + for (SproxydUrl url : urls) { + url.registerEndpoint(endpoints); + } + } + + synchronized SproxydUrl getNext() throws SproxydClientException { + int trials = urls.length; + while (trials > 0) { + current = (current + 1) % urls.length; + final SproxydUrl url = urls[current]; + if (url.isUp()) return url; + trials -= 1; + } + throw new SproxydClientException(0, "All servers are blacklisted"); + } + + @Override + public String toString() { + return Arrays.toString(urls); + } + + } + + private static void info(String format, Object arg) { + if (LOG != null) LOG.info(format, arg); + } + + private static void warn(String format, Object arg) { + if (LOG != null) LOG.warn(format, arg); + } + + private static void warn(String format, Object arg1, Object arg2) { + if (LOG != null) LOG.warn(format, arg1, arg2); + } } diff --git a/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydClientException.java b/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydClientException.java index c6a3b17e..a37a58c5 100644 --- a/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydClientException.java +++ b/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydClientException.java @@ -1,5 +1,23 @@ /** +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -21,28 +39,28 @@ public class SproxydClientException extends HttpResponseException { - private Header[] httpHeaders; - private StatusLine httpStatusLine; + private Header[] httpHeaders; + private StatusLine httpStatusLine; - public SproxydClientException(int code, String message) { - super(code, message); - } + public SproxydClientException(int code, String message) { + super(code, message); + } - public SproxydClientException(int code, String message, Header[] httpHeaders, - StatusLine httpStatusLine) { - super(code, message); - this.httpHeaders = httpHeaders; - this.httpStatusLine = httpStatusLine; - } + public SproxydClientException(int code, String message, Header[] httpHeaders, + StatusLine httpStatusLine) { + super(code, message); + this.httpHeaders = httpHeaders; + this.httpStatusLine = httpStatusLine; + } - public Header[] getHttpHeaders() { - return httpHeaders; - } + public Header[] getHttpHeaders() { + return httpHeaders; + } - public StatusLine getHttpStatusLine() { - return httpStatusLine; - } + public StatusLine getHttpStatusLine() { + return httpStatusLine; + } - private static final long serialVersionUID = 1L; + private static final long serialVersionUID = 1L; } diff --git a/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydConstants.java b/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydConstants.java index ad577af6..805f83bf 100644 --- a/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydConstants.java +++ b/dev/cosbench-scality/src/com/scality/cosbench/client/SproxydConstants.java @@ -1,5 +1,23 @@ /** +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ +/** + Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at @@ -17,30 +35,30 @@ public interface SproxydConstants { - String LOGGING_KEY = "logging"; + String LOGGING_KEY = "logging"; - Boolean LOGGING_DEFAULT = Boolean.FALSE; + Boolean LOGGING_DEFAULT = Boolean.FALSE; - // -------------------------------------------------------------------------- - // CONNECTION - // -------------------------------------------------------------------------- + // -------------------------------------------------------------------------- + // CONNECTION + // -------------------------------------------------------------------------- - String CONN_TIMEOUT_KEY = "timeout"; - int CONN_TIMEOUT_DEFAULT = 30000; + String CONN_TIMEOUT_KEY = "timeout"; + int CONN_TIMEOUT_DEFAULT = 30000; - String BASE_PATH_KEY = "base_path"; - String BASE_PATH_DEFAULT = "/proxy/chord"; + String BASE_PATH_KEY = "base_path"; + String BASE_PATH_DEFAULT = "/proxy/chord"; - String HOSTS_KEY = "hosts"; - String HOSTS_DEFAULT = "127.0.0.1"; + String HOSTS_KEY = "hosts"; + String HOSTS_DEFAULT = "127.0.0.1"; - String PORT_KEY = "port"; - int PORT_DEFAULT = 81; + String PORT_KEY = "port"; + int PORT_DEFAULT = 81; - String POOL_SIZE_KEY = "pool_size"; - /** - * MaxTotal,MaxPerRoute for the HTTP connection manager pool - */ - String POOL_SIZE_DEFAULT = "60,10"; + String POOL_SIZE_KEY = "pool_size"; + /** + * MaxTotal,MaxPerRoute for the HTTP connection manager pool + */ + String POOL_SIZE_DEFAULT = "60,10"; } diff --git a/dev/cosbench-swauth/META-INF/MANIFEST.MF b/dev/cosbench-swauth/META-INF/MANIFEST.MF index c4d87a33..ca09197e 100644 --- a/dev/cosbench-swauth/META-INF/MANIFEST.MF +++ b/dev/cosbench-swauth/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Swauth Client Bundle Bundle-SymbolicName: cosbench-swauth -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuth.java b/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuth.java index 69d45c8b..1e76416b 100644 --- a/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuth.java +++ b/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuth.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.swauth; @@ -33,9 +33,9 @@ /** * This class encapsulates a Swauth implementation for the Auth-API. - * + * * @author ywang19, qzheng7 - * + * */ class SwiftAuth extends NoneAuth { @@ -77,13 +77,13 @@ public void init(Config config, Logger logger) { } @Override public void init() { - super.init(); - logger.debug("a second init of client using auth config: {}",parms); - HttpClient httpClient = HttpClientUtil.createHttpClient(timeout); + super.init(); + logger.debug("a second init of client using auth config: {}",parms); + HttpClient httpClient = HttpClientUtil.createHttpClient(timeout); client = new SwiftAuthClient(httpClient, url, username, password); logger.debug("a second swauth client has been initialized"); } - + @Override public void dispose() { @@ -94,11 +94,11 @@ public void dispose() { @Override public AuthContext login() { super.login(); -// AuthContext authContext = new AuthContext(); -// SwiftTokenCache tokenCache = SwiftTokenCacheImpl.getSwiftTokenCache(client); -// authContext.put("token",tokenCache.getToken()); +// AuthContext authContext = new AuthContext(); +// SwiftTokenCache tokenCache = SwiftTokenCacheImpl.getSwiftTokenCache(client); +// authContext.put("token",tokenCache.getToken()); // authContext.put("storage_url", tokenCache.getStorageURL()); -// return authContext; +// return authContext; try { client.login(); } catch (SwiftAuthClientException se) { @@ -108,10 +108,10 @@ public AuthContext login() { } return createContext(); } - + private AuthContext createContext() { SwiftAuthContext context = new SwiftAuthContext(url, username, password, client.getAuthToken(), client.getStorageURL()); - + return context; } } diff --git a/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuthContext.java b/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuthContext.java index 8448fcec..48c9bb13 100644 --- a/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuthContext.java +++ b/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuthContext.java @@ -1,67 +1,67 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.api.swauth; - -import static com.intel.cosbench.client.swauth.SwiftAuthConstants.*; -import com.intel.cosbench.api.context.AuthContext; - -/** - * This class encapsulates a SwAuth/tempAuth specific AuthContext. - * - * @author ywang19 - * - */ -public class SwiftAuthContext extends AuthContext { - - public SwiftAuthContext(String url, String username, String password, String token, String storage_url) { - this.put(AUTH_URL_KEY, url); - this.put(AUTH_USERNAME_KEY, username); - this.put(AUTH_PASSWORD_KEY, password); - this.put(AUTH_TOKEN_KEY, token); - this.put(STORAGE_URL_KEY, storage_url); - } - - @Override - public String getID(String meta) { - StringBuilder builder = new StringBuilder(); - builder.append(getID()); - builder.append(";meta="); - builder.append(meta); - - return builder.toString(); - } - - @Override - public String getID() { - StringBuilder builder = new StringBuilder(); - builder.append(AUTH_URL_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_URL_KEY)); - builder.append(";"); - builder.append(AUTH_USERNAME_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_USERNAME_KEY)); - builder.append(";"); - builder.append(AUTH_PASSWORD_KEY); - builder.append("="); - builder.append(this.getStr(AUTH_PASSWORD_KEY)); - - return builder.toString(); - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.api.swauth; + +import static com.intel.cosbench.client.swauth.SwiftAuthConstants.*; +import com.intel.cosbench.api.context.AuthContext; + +/** + * This class encapsulates a SwAuth/tempAuth specific AuthContext. + * + * @author ywang19 + * + */ +public class SwiftAuthContext extends AuthContext { + + public SwiftAuthContext(String url, String username, String password, String token, String storage_url) { + this.put(AUTH_URL_KEY, url); + this.put(AUTH_USERNAME_KEY, username); + this.put(AUTH_PASSWORD_KEY, password); + this.put(AUTH_TOKEN_KEY, token); + this.put(STORAGE_URL_KEY, storage_url); + } + + @Override + public String getID(String meta) { + StringBuilder builder = new StringBuilder(); + builder.append(getID()); + builder.append(";meta="); + builder.append(meta); + + return builder.toString(); + } + + @Override + public String getID() { + StringBuilder builder = new StringBuilder(); + builder.append(AUTH_URL_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_URL_KEY)); + builder.append(";"); + builder.append(AUTH_USERNAME_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_USERNAME_KEY)); + builder.append(";"); + builder.append(AUTH_PASSWORD_KEY); + builder.append("="); + builder.append(this.getStr(AUTH_PASSWORD_KEY)); + + return builder.toString(); + } + +} diff --git a/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuthFactory.java b/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuthFactory.java index 31fda89a..1b5c5af9 100644 --- a/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuthFactory.java +++ b/dev/cosbench-swauth/src/com/intel/cosbench/api/swauth/SwiftAuthFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.swauth; diff --git a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthClient.java b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthClient.java index 57bb4296..f50e7619 100644 --- a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthClient.java +++ b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthClient.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swauth; @@ -68,8 +68,8 @@ public void login() throws IOException, SwiftAuthClientException { method.setHeader(X_STORAGE_PASS, password); response = client.execute(method); if ((response.getStatusLine().getStatusCode() >= HttpStatus.SC_OK) && - (response.getStatusLine().getStatusCode() < (HttpStatus.SC_OK + 100)) - ) { + (response.getStatusLine().getStatusCode() < (HttpStatus.SC_OK + 100)) + ) { authToken = response.getFirstHeader(X_AUTH_TOKEN) != null ? response .getFirstHeader(X_AUTH_TOKEN).getValue() : null; storageURL = response.getFirstHeader(X_STORAGE_URL) != null ? response @@ -84,23 +84,23 @@ public void login() throws IOException, SwiftAuthClientException { EntityUtils.consume(response.getEntity()); } } - + public boolean check() { - HttpResponse response; - if(storageURL == null || authToken == null) - return false; - try { - HttpHead method = new HttpHead(storageURL); - method.setHeader(X_AUTH_TOKEN, authToken); - response = client.execute(method); - - if ((response.getStatusLine().getStatusCode() >= HttpStatus.SC_OK) && - (response.getStatusLine().getStatusCode() < (HttpStatus.SC_OK + 100))) - return true; - } catch (IOException e) { - } - return false; + HttpResponse response; + if(storageURL == null || authToken == null) + return false; + try { + HttpHead method = new HttpHead(storageURL); + method.setHeader(X_AUTH_TOKEN, authToken); + response = client.execute(method); + + if ((response.getStatusLine().getStatusCode() >= HttpStatus.SC_OK) && + (response.getStatusLine().getStatusCode() < (HttpStatus.SC_OK + 100))) + return true; + } catch (IOException e) { + } + return false; } - + } diff --git a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthClientException.java b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthClientException.java index 87d9a3f0..1c7b4736 100644 --- a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthClientException.java +++ b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthClientException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swauth; @@ -27,9 +27,9 @@ public class SwiftAuthClientException extends HttpResponseException { private StatusLine httpStatusLine; public SwiftAuthClientException(int code, String message) { - super(code, message); + super(code, message); } - + public SwiftAuthClientException(int code, String message, Header[] httpHeaders, StatusLine httpStatusLine) { super(code, message); @@ -44,7 +44,7 @@ public Header[] getHttpHeaders() { public StatusLine getHttpStatusLine() { return httpStatusLine; } - + private static final long serialVersionUID = 1L; } diff --git a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthConstants.java b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthConstants.java index 80728c81..86098939 100644 --- a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthConstants.java +++ b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/SwiftAuthConstants.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swauth; diff --git a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/utils/SwiftTokenCache.java b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/utils/SwiftTokenCache.java index 804f03e3..a8ca81ab 100644 --- a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/utils/SwiftTokenCache.java +++ b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/utils/SwiftTokenCache.java @@ -1,48 +1,66 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.client.swauth.utils; public class SwiftTokenCache { - private String token; - private String storageURL; - private long version; + private String token; + private String storageURL; + private long version; + + + public SwiftTokenCache() + { + token = "dummyToken"; + storageURL = "dummyStorageURL"; + version = 0; + } + + public String getToken() { + return token; + } - - public SwiftTokenCache() - { - token = "dummyToken"; - storageURL = "dummyStorageURL"; - version = 0; - } - - public String getToken() { - return token; - } + public void setToken(String token) { + this.token = token; + } - public void setToken(String token) { - this.token = token; - } + public String getStorageURL() { + return storageURL; + } - public String getStorageURL() { - return storageURL; - } + public void setStorageURL(String storageURL) { + this.storageURL = storageURL; + } - public void setStorageURL(String storageURL) { - this.storageURL = storageURL; - } + public long getVersion() { + return version; + } - public long getVersion() { - return version; - } + public void setVersion(long version) { + this.version = version; + } - public void setVersion(long version) { - this.version = version; - } - - public void incrementVersion() { - this.version++; - } + public void incrementVersion() { + this.version++; + } } diff --git a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/utils/SwiftTokenCacheImpl.java b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/utils/SwiftTokenCacheImpl.java index f41e6f5a..18c311d1 100644 --- a/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/utils/SwiftTokenCacheImpl.java +++ b/dev/cosbench-swauth/src/com/intel/cosbench/client/swauth/utils/SwiftTokenCacheImpl.java @@ -1,3 +1,21 @@ +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. +Copyright 2019 OpenIO Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +*/ package com.intel.cosbench.client.swauth.utils; import java.io.InterruptedIOException; @@ -12,36 +30,36 @@ import com.intel.cosbench.client.swauth.SwiftAuthClientException; public class SwiftTokenCacheImpl { - - private static SwiftTokenCache latestTokenCache = new SwiftTokenCache(); - + private static SwiftTokenCache latestTokenCache = new SwiftTokenCache(); + + public static synchronized void loadSwiftTokenCache(SwiftAuthClient client, SwiftTokenCache currentTokenCache) { // Actual operation to load token cache from server - if(currentTokenCache.getVersion() == latestTokenCache.getVersion()) - { - try { - client.login(); - latestTokenCache.setToken(client.getAuthToken()); - latestTokenCache.setStorageURL(client.getStorageURL()); - latestTokenCache.incrementVersion(); - } catch (SocketTimeoutException ste) { - throw new AuthTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new AuthTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new AuthInterruptedException(ie); - } catch (SwiftAuthClientException se) { - throw new AuthException(se.getMessage(), se); - } catch (Exception e) { - throw new AuthException(e); - } + if(currentTokenCache.getVersion() == latestTokenCache.getVersion()) + { + try { + client.login(); + latestTokenCache.setToken(client.getAuthToken()); + latestTokenCache.setStorageURL(client.getStorageURL()); + latestTokenCache.incrementVersion(); + } catch (SocketTimeoutException ste) { + throw new AuthTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new AuthTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new AuthInterruptedException(ie); + } catch (SwiftAuthClientException se) { + throw new AuthException(se.getMessage(), se); + } catch (Exception e) { + throw new AuthException(e); } - } + } + } public static SwiftTokenCache getSwiftTokenCache(SwiftAuthClient client) { - if(latestTokenCache.getVersion()==0 || ! client.check()) - loadSwiftTokenCache(client,latestTokenCache); - return latestTokenCache; + if(latestTokenCache.getVersion()==0 || ! client.check()) + loadSwiftTokenCache(client,latestTokenCache); + return latestTokenCache; } } diff --git a/dev/cosbench-swift/META-INF/MANIFEST.MF b/dev/cosbench-swift/META-INF/MANIFEST.MF index 815d0057..08169cef 100644 --- a/dev/cosbench-swift/META-INF/MANIFEST.MF +++ b/dev/cosbench-swift/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Swift Client Bundle Bundle-SymbolicName: cosbench-swift -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Vendor: intel Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Import-Package: com.intel.cosbench.api.auth, diff --git a/dev/cosbench-swift/src/com/intel/cosbench/api/swift/SwiftStorage.java b/dev/cosbench-swift/src/com/intel/cosbench/api/swift/SwiftStorage.java index 93f80c45..c9468896 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/api/swift/SwiftStorage.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/api/swift/SwiftStorage.java @@ -1,308 +1,311 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.api.swift; - -import static com.intel.cosbench.client.swift.SwiftConstants.*; - -import java.io.*; -import java.net.SocketTimeoutException; -import java.util.Map; - -import org.apache.http.client.HttpClient; -import org.apache.http.conn.ConnectTimeoutException; - -import com.intel.cosbench.api.context.AuthContext; -import com.intel.cosbench.api.context.DefaultAuthContext; -import com.intel.cosbench.api.storage.*; -import com.intel.cosbench.client.http.HttpClientUtil; -import com.intel.cosbench.client.swift.*; -import com.intel.cosbench.config.Config; -import com.intel.cosbench.log.Logger; - -/** - * This class encapsulates a Swift implementation for Storage API. - * - * @author ywang19, qzheng7 - * - */ -class SwiftStorage extends NoneStorage { - - private SwiftClient client; - - /* configurations */ - private int timeout; // connection and socket timeout - private String token; - private String storage_url; - private String policy; - - public SwiftStorage() { - /* empty */ - } - - @Override - public void init(Config config, Logger logger) { - super.init(config, logger); - - timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); - token = config.get(AUTH_TOKEN_KEY, AUTH_TOKEN_DEFAULT); - storage_url = config.get(STORAGE_URL_KEY, STORAGE_URL_DEFAULT); - policy = config.get(POLICY_KEY, POLICY_DEFAULT); - - parms.put(CONN_TIMEOUT_KEY, timeout); - parms.put(AUTH_TOKEN_KEY, token); - parms.put(STORAGE_URL_KEY, storage_url); - parms.put(POLICY_KEY, policy); - - logger.debug("using storage config: {}", parms); - - HttpClient httpClient = HttpClientUtil.createHttpClient(timeout); - - client = new SwiftClient(httpClient); - logger.debug("swift client has been initialized"); - } - - @Override - public void setAuthContext(AuthContext info) { - super.setAuthContext(info); - - if(info != null) { - token = info.getStr(AUTH_TOKEN_KEY); - storage_url = info.getStr(STORAGE_URL_KEY); - } - - try { - client.init(token, storage_url, policy); - } catch (Exception e) { - throw new StorageException(e); - } - logger.debug(new StringBuffer() - .append("using auth token: ") - .append(token) - .append(", storage url: ") - .append(storage_url) - .append(", storage policy: ") - .append(policy).toString()); - } - - @Override - public AuthContext getAuthContext() { - String token = client.getAuthToken(); - String storage_url = client.getStorageURL(); - - AuthContext info = new DefaultAuthContext(); - info.put(AUTH_TOKEN_KEY, token); - info.put(STORAGE_URL_KEY, storage_url); - - logger.debug("returned auth token: {}, storage url: {}", token, storage_url); - return info; - } - - @Override - public void dispose() { - super.dispose(); - client.dispose(); - } - - @Override - public void abort() { - super.abort(); - client.abort(); - } - - @Override - public InputStream getObject(String container, String object, Config config) { - super.getObject(container, object, config); - InputStream stream; - try { - stream = client.getObjectAsStream(container, object); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - return stream; - } - - @Override - public InputStream getList(String container, String object, Config config) { - super.getList(container, object, config); - InputStream stream; - try { - stream = client.getTargetList(container, object); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - return stream; - } - - @Override - public void createContainer(String container, Config config) { - super.createContainer(container, config); - try { - if (!client.containerExists(container)) - client.createContainer(container); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Deprecated - public void createObject(String container, String object, byte[] data, - Config config) { - super.createObject(container, object, data, config); - try { - client.storeObject(container, object, data); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void createObject(String container, String object, InputStream data, - long length, Config config) { - super.createObject(container, object, data, length, config); - try { - client.storeStreamedObject(container, object, data, length); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void deleteContainer(String container, Config config) { - super.deleteContainer(container, config); - try { - if (client.containerExists(container)) - client.deleteContainer(container); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - public void deleteObject(String container, String object, Config config) { - super.deleteObject(container, object, config); - try { - client.deleteObject(container, object); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - protected void createMetadata(String container, String object, - Map map, Config config) { - super.createMetadata(container, object, map, config); - try { - client.storeObjectMetadata(container, object, map); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - - @Override - protected Map getMetadata(String container, String object, - Config config) { - super.getMetadata(container, object, config); - try { - return client.getObjectMetadata(container, object); - } catch (SocketTimeoutException ste) { - throw new StorageTimeoutException(ste); - } catch (ConnectTimeoutException cte) { - throw new StorageTimeoutException(cte); - } catch (InterruptedIOException ie) { - throw new StorageInterruptedException(ie); - } catch (SwiftException se) { - String msg = se.getHttpStatusLine().toString(); - throw new StorageException(msg, se); - } catch (Exception e) { - throw new StorageException(e); - } - } - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.api.swift; + +import static com.intel.cosbench.client.swift.SwiftConstants.*; + +import java.io.*; +import java.net.SocketTimeoutException; +import java.util.Map; + +import org.apache.http.client.HttpClient; +import org.apache.http.conn.ConnectTimeoutException; + +import com.intel.cosbench.api.context.AuthContext; +import com.intel.cosbench.api.context.DefaultAuthContext; +import com.intel.cosbench.api.storage.*; +import com.intel.cosbench.client.http.HttpClientUtil; +import com.intel.cosbench.client.swift.*; +import com.intel.cosbench.config.Config; +import com.intel.cosbench.log.Logger; + +/** + * This class encapsulates a Swift implementation for Storage API. + * + * @author ywang19, qzheng7 + * + */ +class SwiftStorage extends NoneStorage { + + private SwiftClient client; + + /* configurations */ + private int timeout; // connection and socket timeout + private String token; + private String storage_url; + private String policy; + private int transfer_rate; + + public SwiftStorage() { + /* empty */ + } + + @Override + public void init(Config config, Logger logger) { + super.init(config, logger); + + timeout = config.getInt(CONN_TIMEOUT_KEY, CONN_TIMEOUT_DEFAULT); + token = config.get(AUTH_TOKEN_KEY, AUTH_TOKEN_DEFAULT); + storage_url = config.get(STORAGE_URL_KEY, STORAGE_URL_DEFAULT); + policy = config.get(POLICY_KEY, POLICY_DEFAULT); + transfer_rate = config.getInt(TRANSFER_RATE, TRANSFER_RATE_DEFAULT); + + parms.put(CONN_TIMEOUT_KEY, timeout); + parms.put(AUTH_TOKEN_KEY, token); + parms.put(STORAGE_URL_KEY, storage_url); + parms.put(POLICY_KEY, policy); + parms.put(TRANSFER_RATE, transfer_rate); + + logger.debug("using storage config: {}", parms); + + HttpClient httpClient = HttpClientUtil.createHttpClient(timeout); + + client = new SwiftClient(httpClient); + logger.debug("swift client has been initialized"); + } + + @Override + public void setAuthContext(AuthContext info) { + super.setAuthContext(info); + + if(info != null) { + token = info.getStr(AUTH_TOKEN_KEY); + storage_url = info.getStr(STORAGE_URL_KEY); + } + + try { + client.init(token, storage_url, policy, transfer_rate); + } catch (Exception e) { + throw new StorageException(e); + } + logger.debug(new StringBuffer() + .append("using auth token: ") + .append(token) + .append(", storage url: ") + .append(storage_url) + .append(", storage policy: ") + .append(policy).toString()); + } + + @Override + public AuthContext getAuthContext() { + String token = client.getAuthToken(); + String storage_url = client.getStorageURL(); + + AuthContext info = new DefaultAuthContext(); + info.put(AUTH_TOKEN_KEY, token); + info.put(STORAGE_URL_KEY, storage_url); + + logger.debug("returned auth token: {}, storage url: {}", token, storage_url); + return info; + } + + @Override + public void dispose() { + super.dispose(); + client.dispose(); + } + + @Override + public void abort() { + super.abort(); + client.abort(); + } + + @Override + public InputStream getObject(String container, String object, Config config) { + super.getObject(container, object, config); + InputStream stream; + try { + stream = client.getObjectAsStream(container, object); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + return stream; + } + + @Override + public InputStream getList(String container, String object, Config config) { + super.getList(container, object, config); + InputStream stream; + try { + stream = client.getTargetList(container, object); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + return stream; + } + + @Override + public void createContainer(String container, Config config) { + super.createContainer(container, config); + try { + if (!client.containerExists(container)) + client.createContainer(container); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Deprecated + public void createObject(String container, String object, byte[] data, + Config config) { + super.createObject(container, object, data, config); + try { + client.storeObject(container, object, data); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void createObject(String container, String object, InputStream data, + long length, Config config) { + super.createObject(container, object, data, length, config); + try { + client.storeStreamedObject(container, object, data, length); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void deleteContainer(String container, Config config) { + super.deleteContainer(container, config); + try { + if (client.containerExists(container)) + client.deleteContainer(container); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + public void deleteObject(String container, String object, Config config) { + super.deleteObject(container, object, config); + try { + client.deleteObject(container, object); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + protected void createMetadata(String container, String object, + Map map, Config config) { + super.createMetadata(container, object, map, config); + try { + client.storeObjectMetadata(container, object, map); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + + @Override + protected Map getMetadata(String container, String object, + Config config) { + super.getMetadata(container, object, config); + try { + return client.getObjectMetadata(container, object); + } catch (SocketTimeoutException ste) { + throw new StorageTimeoutException(ste); + } catch (ConnectTimeoutException cte) { + throw new StorageTimeoutException(cte); + } catch (InterruptedIOException ie) { + throw new StorageInterruptedException(ie); + } catch (SwiftException se) { + String msg = se.getHttpStatusLine().toString(); + throw new StorageException(msg, se); + } catch (Exception e) { + throw new StorageException(e); + } + } + +} diff --git a/dev/cosbench-swift/src/com/intel/cosbench/api/swift/SwiftStorageFactory.java b/dev/cosbench-swift/src/com/intel/cosbench/api/swift/SwiftStorageFactory.java index c45f08be..9f8e56a9 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/api/swift/SwiftStorageFactory.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/api/swift/SwiftStorageFactory.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.api.swift; diff --git a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftAccount.java b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftAccount.java index bab9b9ac..8b7ae9ba 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftAccount.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftAccount.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swift; diff --git a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftClient.java b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftClient.java index 02ec1c7a..072731b2 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftClient.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftClient.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swift; @@ -32,12 +32,13 @@ public class SwiftClient { - private static boolean REPORT_DELETE_ERROR = false; + private static boolean REPORT_DELETE_ERROR = false; /* user context */ private String authToken; private String storageURL; private String policy; + private int rate; /* HTTP client */ private HttpClient client; @@ -68,10 +69,11 @@ public void abort() { method = null; } - public void init(String authToken, String storageURL, String policy) { + public void init(String authToken, String storageURL, String policy, int rate) { this.authToken = authToken; this.storageURL = storageURL; this.policy = policy; + this.rate = rate; } public SwiftAccount getAccountInfo() throws IOException, SwiftException { @@ -121,20 +123,20 @@ public void createContainer(String container) throws IOException, SwiftException { SwiftResponse response = null; try { - Logger logger = LogFactory.getSystemLogger(); - logger.debug("Creating container with auth_token " + authToken); + Logger logger = LogFactory.getSystemLogger(); + logger.debug("Creating container with auth_token " + authToken); method = HttpClientUtil.makeHttpPut(getContainerPath(container)); method.setHeader(X_AUTH_TOKEN, authToken); if(policy != null) - method.setHeader(X_STORAGE_POLICY, policy); + method.setHeader(X_STORAGE_POLICY, policy); response = new SwiftResponse(client.execute(method)); if (response.getStatusCode() == SC_CREATED) { - logger.debug("Creating container "+container+" SUCCESS"); + logger.debug("Creating container "+container+" SUCCESS"); return; } if (response.getStatusCode() == SC_ACCEPTED) { - logger.debug("Creating container "+container+" SUCCESS"); + logger.debug("Creating container "+container+" SUCCESS"); return; } throw new SwiftException("unexpected return from server", @@ -174,6 +176,7 @@ public InputStream getObjectAsStream(String container, String object) throws IOException, SwiftException { method = HttpClientUtil.makeHttpGet(getObjectPath(container, object)); method.setHeader(X_AUTH_TOKEN, authToken); + method.setHeader(X_TRANSFER_RATE, new Integer(rate).toString()); SwiftResponse response = new SwiftResponse(client.execute(method)); if (response.getStatusCode() == SC_OK) return response.getResponseBodyAsStream(); @@ -184,28 +187,28 @@ public InputStream getObjectAsStream(String container, String object) throw new SwiftException("unexpected result from server", response.getResponseHeaders(), response.getStatusLine()); } - + public InputStream getTargetList(String container, String object) throws IOException, SwiftException { - if (object.isEmpty()) - method = HttpClientUtil.makeHttpGet(getObjectPath(container, object)); - else - method = HttpClientUtil.makeHttpHead(getObjectPath(container, object)); + if (object.isEmpty()) + method = HttpClientUtil.makeHttpGet(getObjectPath(container, object)); + else + method = HttpClientUtil.makeHttpHead(getObjectPath(container, object)); method.setHeader(X_AUTH_TOKEN, authToken); SwiftResponse response = new SwiftResponse(client.execute(method)); - + if (response.getStatusCode() == SC_OK) { - if (!object.isEmpty() && response != null) - response.consumeResposeBody(); + if (!object.isEmpty() && response != null) + response.consumeResposeBody(); return object.isEmpty() ? response.getResponseBodyAsStream() - : new ByteArrayInputStream(new byte[] {}); + : new ByteArrayInputStream(new byte[] {}); } response.consumeResposeBody(); if (response.getStatusCode() == SC_NOT_FOUND) - throw new SwiftFileNotFoundException("list target not found " + container + " / " - + object, response.getResponseHeaders(), response.getStatusLine()); + throw new SwiftFileNotFoundException("list target not found " + container + " / " + + object, response.getResponseHeaders(), response.getStatusLine()); throw new SwiftException("unexpected result from server", response.getResponseHeaders(), response.getStatusLine()); - } + } public void storeObject(String container, String object, byte[] data) throws IOException, SwiftException { diff --git a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftConflictException.java b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftConflictException.java index 521eeb6c..828ff9ee 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftConflictException.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftConflictException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swift; diff --git a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftConstants.java b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftConstants.java index 648d954f..e7f47429 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftConstants.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftConstants.java @@ -1,61 +1,66 @@ -/** - -Copyright 2013 Intel Corporation, All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package com.intel.cosbench.client.swift; - -public interface SwiftConstants { - - // -------------------------------------------------------------------------- - // CONNECTION - // -------------------------------------------------------------------------- - - String CONN_TIMEOUT_KEY = "timeout"; - - Integer CONN_TIMEOUT_DEFAULT = 30000; - - // -------------------------------------------------------------------------- - // CONTEXT NEEDS FROM AUTH MODULE - // -------------------------------------------------------------------------- - - String AUTH_TOKEN_KEY = "token"; - String AUTH_TOKEN_DEFAULT = "AUTH_xxx"; - - String STORAGE_URL_KEY = "storage_url"; - String STORAGE_URL_DEFAULT = "http://127.0.0.1:8080/auth/v1.0"; - - String POLICY_KEY = "policy"; - String POLICY_DEFAULT = null; - - // -------------------------------------------------------------------------- - // Swift RESTful API - // -------------------------------------------------------------------------- - - String X_STORAGE_URL = "X-Storage-Url"; - - String X_AUTH_TOKEN = "X-Auth-Token"; - - String X_STORAGE_POLICY = "X-Storage-Policy"; - - String X_CONTAINER_OBJECT_COUNT = "X-Container-Object-Count"; - - String X_CONTAINER_BYTES_USED = "X-Container-Bytes-Used"; - - String X_ACCOUNT_CONTAINER_COUNT = "X-Account-Container-Count"; - - String X_ACCOUNT_BYTES_USED = "X-Account-Bytes-Used"; - -} +/** + +Copyright 2013 Intel Corporation, All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package com.intel.cosbench.client.swift; + +public interface SwiftConstants { + + // -------------------------------------------------------------------------- + // CONNECTION + // -------------------------------------------------------------------------- + + String CONN_TIMEOUT_KEY = "timeout"; + + Integer CONN_TIMEOUT_DEFAULT = 30000; + + // -------------------------------------------------------------------------- + // CONTEXT NEEDS FROM AUTH MODULE + // -------------------------------------------------------------------------- + + String AUTH_TOKEN_KEY = "token"; + String AUTH_TOKEN_DEFAULT = "AUTH_xxx"; + + String STORAGE_URL_KEY = "storage_url"; + String STORAGE_URL_DEFAULT = "http://127.0.0.1:8080/auth/v1.0"; + + String POLICY_KEY = "policy"; + String POLICY_DEFAULT = null; + + String TRANSFER_RATE = "transfer_rate"; + Integer TRANSFER_RATE_DEFAULT = 50000000; + + // -------------------------------------------------------------------------- + // Swift RESTful API + // -------------------------------------------------------------------------- + + String X_STORAGE_URL = "X-Storage-Url"; + + String X_AUTH_TOKEN = "X-Auth-Token"; + + String X_STORAGE_POLICY = "X-Storage-Policy"; + + String X_CONTAINER_OBJECT_COUNT = "X-Container-Object-Count"; + + String X_CONTAINER_BYTES_USED = "X-Container-Bytes-Used"; + + String X_ACCOUNT_CONTAINER_COUNT = "X-Account-Container-Count"; + + String X_ACCOUNT_BYTES_USED = "X-Account-Bytes-Used"; + + String X_TRANSFER_RATE = "X-Transfer-Rate"; + +} diff --git a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftContainer.java b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftContainer.java index 33c2db87..84ebd64a 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftContainer.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftContainer.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swift; diff --git a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftException.java b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftException.java index 0fdf61c6..78fb0a5e 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftException.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swift; diff --git a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftFileNotFoundException.java b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftFileNotFoundException.java index 943410e1..41730f73 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftFileNotFoundException.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftFileNotFoundException.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swift; diff --git a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftResponse.java b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftResponse.java index 55506bcf..83e4ad7f 100644 --- a/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftResponse.java +++ b/dev/cosbench-swift/src/com/intel/cosbench/client/swift/SwiftResponse.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.client.swift; @@ -27,7 +27,7 @@ /** * This class encapsulates one wrapper of http response. - * + * * @author ywang19, qzheng7 * */ diff --git a/dev/cosbench-tomcat/META-INF/MANIFEST.MF b/dev/cosbench-tomcat/META-INF/MANIFEST.MF index 41da3411..a0631207 100644 --- a/dev/cosbench-tomcat/META-INF/MANIFEST.MF +++ b/dev/cosbench-tomcat/META-INF/MANIFEST.MF @@ -2,7 +2,7 @@ Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: COSBench Tomcat Bundle Bundle-SymbolicName: cosbench-tomcat -Bundle-Version: 0.4.1.0 +Bundle-Version: 0.4.7 Bundle-Activator: com.intel.cosbench.tomcat.osgi.Activator Import-Package: com.intel.cosbench.log, org.apache.catalina;version="[6.0.16,7.0.0)", diff --git a/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/TomcatLoader.java b/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/TomcatLoader.java index 2662ac92..0322131d 100644 --- a/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/TomcatLoader.java +++ b/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/TomcatLoader.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.tomcat; @@ -30,9 +30,9 @@ /** * The class encapsulates tomcat configuration loading and catalina starting up. - * + * * @author ywang19, qzheng7 - * + * */ public class TomcatLoader extends Catalina { @@ -62,7 +62,7 @@ public TomcatLoader() { * folder, 4. predefined default configuration file: - on unix like * environment, it's "/etc/cosbench/tomcat-server.xml" - on windows * environment, it's "c:\\tomcat-server.xml" - * + * * @return the path to tomcat configuration file */ private static String getServerConfig() { @@ -81,7 +81,7 @@ private static String getServerConfig() { *

* This method will start up tomcat service, and show listening port * information on console. - * + * */ @Override public void start() { diff --git a/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/TomcatRegister.java b/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/TomcatRegister.java index a5f6925a..9253c2ae 100644 --- a/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/TomcatRegister.java +++ b/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/TomcatRegister.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.tomcat; @@ -27,9 +27,9 @@ /** * The class encapsulates the registration of service exposed by Tomcat bundle. - * + * * @author ywang19, qzheng7 - * + * */ public class TomcatRegister { @@ -46,7 +46,7 @@ public TomcatRegister() { /** * This method will register service into osgi context, so other bundles can * access the service. - * + * * @param context * the osgi bundle context * @param service @@ -59,7 +59,7 @@ public void publishService(BundleContext context, StandardService service) { /** * This method will unregister service from osgi context. - * + * */ public void unpublishService() { registration.unregister(); diff --git a/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/osgi/Activator.java b/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/osgi/Activator.java index a090bea3..358a70d9 100644 --- a/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/osgi/Activator.java +++ b/dev/cosbench-tomcat/src/com/intel/cosbench/tomcat/osgi/Activator.java @@ -1,5 +1,5 @@ -/** - +/** + Copyright 2013 Intel Corporation, All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,8 +12,8 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and -limitations under the License. -*/ +limitations under the License. +*/ package com.intel.cosbench.tomcat.osgi; @@ -24,9 +24,9 @@ /** * The bundle activator for tomcat. - * + * * @author ywang19, qzheng7 - * + * */ public class Activator implements BundleActivator { @@ -53,7 +53,7 @@ public class Activator implements BundleActivator { * * * - * + * * @return the path to tomcat configuration file * @see org.osgi.framework.BundleActivator#start(org.osgi.framework.BundleContext) */ diff --git a/ext/adaptor/abc-auth/src/com/abc/api/abcAuth/AbcAuth.java b/ext/adaptor/abc-auth/src/com/abc/api/abcAuth/AbcAuth.java index 92951717..bfdd9dfa 100644 --- a/ext/adaptor/abc-auth/src/com/abc/api/abcAuth/AbcAuth.java +++ b/ext/adaptor/abc-auth/src/com/abc/api/abcAuth/AbcAuth.java @@ -87,7 +87,7 @@ public AuthContext login() { } private AuthContext createContext() { - AuthContext context = new AuthContext(); + AuthContext context = new DefaultAuthContext(); context.put(AUTH_TOKEN_KEY, client.getAuthToken()); return context; } diff --git a/pack-openio.sh b/pack-openio.sh new file mode 100755 index 00000000..bf2907fb --- /dev/null +++ b/pack-openio.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# +#Copyright 2019 OpenIO Corporation, All Rights Reserved. +# +#Licensed under the Apache License, Version 2.0 (the "License"); +#you may not use this file except in compliance with the License. +#You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +#Unless required by applicable law or agreed to in writing, software +#distributed under the License is distributed on an "AS IS" BASIS, +#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +#See the License for the specific language governing permissions and +#limitations under the License. +# + +PREFIX="$(dirname $0)" +VERSION=$(cat VERSION) +DIST="$PREFIX/$VERSION" +DIST_OPENIO="cosbench-$VERSION-openio" + + +if test -d $DIST; then + echo -n "Deleting old $DIST ... " + rm -rf $DIST + echo "[OK]" +fi + +echo -n "calling $PREFIX/pack.sh for version $VERSION ... " +OUT="$($PREFIX/pack.sh $VERSION 2>&1)" +echo "[OK]:" +echo "$OUT" + +echo -n "Moving $DIST to $PREFIX/$DIST_OPENIO ... " +mv $DIST $PREFIX/$DIST_OPENIO +echo "[OK]" + +echo -n "Creating archive $PREFIX/$DIST_OPENIO.tar.gz ... " +tar -C $PREFIX -czf $DIST_OPENIO.tar.gz $DIST_OPENIO +echo "[OK]" + +echo -n "Removing $PREFIX/$DIST_OPENIO ... " +rm -rf $PREFIX/$DIST_OPENIO +echo "[OK]" + +echo "DONE -> Cosbench OpenIO $VERSION generated in $PREFIX/$DIST_OPENIO.tar.gz!" +exit 0 diff --git a/pack.cmd b/pack.cmd index bd589dd4..5cf64433 100644 --- a/pack.cmd +++ b/pack.cmd @@ -45,6 +45,9 @@ xcopy /Y COSBenchAdaptorDevGuide.pdf %1 xcopy /Y 3rd-party-licenses.pdf %1 xcopy /Y pkg.lst %1 +type nul>%1/BUILD.no +cscript /nologo datescript.vbs> %1/BUILD.no + echo "Build up adaptor example enviornment" xcopy /Y /E ext\* %1\ext\ diff --git a/pack.sh b/pack.sh index dbec728a..e4677433 100755 --- a/pack.sh +++ b/pack.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # #Copyright 2013 Intel Corporation, All Rights Reserved. # @@ -43,6 +43,8 @@ cp -f 3rd-party-licenses.pdf $1/ cp -f pkg.lst $1/ -echo "Build up adaptor example enviornment" +echo $(date +%Y%m%d) > $1/BUILD.no + +echo "Build up adaptor example environment" mkdir $1/ext cp -f -R ext/* $1/ext diff --git a/release/BUILD.no b/release/BUILD.no new file mode 100644 index 00000000..9faa01c3 --- /dev/null +++ b/release/BUILD.no @@ -0,0 +1 @@ +20150318 diff --git a/release/VERSION b/release/VERSION new file mode 100644 index 00000000..f7abe273 --- /dev/null +++ b/release/VERSION @@ -0,0 +1 @@ +0.4.2 \ No newline at end of file diff --git a/release/conf/controller-tomcat-server.xml b/release/conf/controller-tomcat-server.xml index 37b82a25..f4b3f6f6 100644 --- a/release/conf/controller-tomcat-server.xml +++ b/release/conf/controller-tomcat-server.xml @@ -1,6 +1,6 @@ - + @@ -9,4 +9,4 @@ xmlNamespaceAware="false" /> - \ No newline at end of file + diff --git a/release/conf/driver-tomcat-server.xml b/release/conf/driver-tomcat-server.xml index cc37fe5e..0305b518 100644 --- a/release/conf/driver-tomcat-server.xml +++ b/release/conf/driver-tomcat-server.xml @@ -1,6 +1,6 @@ - + @@ -9,4 +9,4 @@ xmlNamespaceAware="false" /> - \ No newline at end of file + diff --git a/release/conf/driver-tomcat-server_template.xml b/release/conf/driver-tomcat-server_template.xml new file mode 100644 index 00000000..0305b518 --- /dev/null +++ b/release/conf/driver-tomcat-server_template.xml @@ -0,0 +1,12 @@ + + + + + + + + + diff --git a/release/conf/driver.conf b/release/conf/driver.conf index 7338ffe8..eb8865bf 100644 --- a/release/conf/driver.conf +++ b/release/conf/driver.conf @@ -1,2 +1,4 @@ [driver] -log_level = INFO \ No newline at end of file +log_level = INFO +log_file = log/system.log +mission_dir = log/mission diff --git a/release/conf/driver_template.conf b/release/conf/driver_template.conf new file mode 100644 index 00000000..5c920b1f --- /dev/null +++ b/release/conf/driver_template.conf @@ -0,0 +1,3 @@ +[driver] +name=driver +url=http://127.0.0.1:18088/driver diff --git a/release/conf/openio-config-sample.xml b/release/conf/openio-config-sample.xml index cbbcb668..780a5725 100644 --- a/release/conf/openio-config-sample.xml +++ b/release/conf/openio-config-sample.xml @@ -1,40 +1,40 @@ - + - + - + - + - + - + - + - + - + - + - + diff --git a/release/conf/swift-config-sample.xml b/release/conf/swift-config-sample.xml index bae71f7e..1b10cc37 100644 --- a/release/conf/swift-config-sample.xml +++ b/release/conf/swift-config-sample.xml @@ -11,7 +11,7 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/release/workloads/zmanda/download-small_(Online gaming hosting service).xml b/release/workloads/zmanda/download-small_(Online gaming hosting service).xml new file mode 100644 index 00000000..b3934c1a --- /dev/null +++ b/release/workloads/zmanda/download-small_(Online gaming hosting service).xml @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/release/workloads/zmanda/upload-large_(Enterprise backup).xml b/release/workloads/zmanda/upload-large_(Enterprise backup).xml new file mode 100644 index 00000000..dbd2754f --- /dev/null +++ b/release/workloads/zmanda/upload-large_(Enterprise backup).xml @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/release/workloads/zmanda/upload-small_(Online gaming hosting service).xml b/release/workloads/zmanda/upload-small_(Online gaming hosting service).xml new file mode 100644 index 00000000..13309717 --- /dev/null +++ b/release/workloads/zmanda/upload-small_(Online gaming hosting service).xml @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/version_unify.sh b/version_unify.sh new file mode 100755 index 00000000..0e90d838 --- /dev/null +++ b/version_unify.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash +release_id=`cat VERSION` +echo "release id is: "$release_id +for dir in `ls dev` +do + f='dev/'$dir"/META-INF/MANIFEST.MF" + if [ -f $f ] + then + sed -i "s/Bundle-Version:.*$/Bundle-Version: ${release_id}/" $f + echo "file: "$f" is updated" + fi +done +