Skip to content

Commit 53a60d1

Browse files
committed
Clean up LocalGcdHelper and docs (#821)
* Cleans up local gcd helper and docs to be more consistent with other test helpers * Renames LocalGcdHelper to LocalDatastoreHelper and RemoteGcsHelper to RemoteStorageHelper * Updates the gcd.sh script version
1 parent 6987b00 commit 53a60d1

File tree

16 files changed

+254
-314
lines changed

16 files changed

+254
-314
lines changed

TESTING.md

Lines changed: 34 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -11,55 +11,64 @@ This library provides tools to help write tests for code that uses the following
1111

1212
#### On your machine
1313

14-
You can test against a temporary local datastore by following these steps:
14+
You can test against a temporary local Datastore by following these steps:
1515

16-
1. Start the local datastore emulator using `LocalGcdHelper`. This can be done in two ways:
17-
- Run `LocalGcdHelper.java`'s `main` method with arguments `START` and (optionally) `--port=<port number>`. This will create a temporary folder on your computer and bind `localhost:<port number>` for communication with the local datastore. The port number is an optional argument. If no port number is specified, port 8080 will be used.
18-
- Call `LocalGcdHelper.start(<project ID>, <port number>)` before running your tests. Save the `LocalGcdHelper` object returned so that you can stop the emulator later.
16+
1. Start the local Datastore emulator before running your tests using `LocalDatastoreHelper`'s `create` and `start` methods. This will create a temporary folder on your computer and bind a port for communication with the local Datastore. There is an optional argument for `create`: consistency. The consistency setting controls the fraction of Datastore writes that are immediately visible in global queries.
17+
```java
18+
// Use the default consistency setting of 0.9
19+
LocalDatastoreHelper helper = LocalDatastoreHelper.create();
20+
// or explicitly set the consistency
21+
helper = LocalDatastoreHelper.create(0.6);
22+
23+
helper.start(); // Starts the local Datastore emulator in a separate process
24+
```
1925

20-
2. In your program, create and use a datastore whose host is set host to `localhost:<port number>`. For example,
26+
2. Create and use a `Datastore` object with the options given by the `LocalDatastoreHelper` instance. For example:
2127
```java
22-
DatastoreOptions options = DatastoreOptions.builder()
23-
.projectId(PROJECT_ID)
24-
.host("http://localhost:8080")
25-
.build();
26-
Datastore localDatastore = options.service();
28+
Datastore localDatastore = helper.options().service();
2729
```
30+
2831
3. Run your tests.
2932

30-
4. Stop the local datastore emulator.
31-
- If you ran `LocalGcdHelper.java`'s `main` function to start the emulator, run `LocalGcdHelper.java`'s `main` method with arguments `STOP` and (optionally) `--port=<port number>`. If the port is not supplied, the program will attempt to close the last port started.
32-
- If you ran `LocalGcdHelper.start()` to start the emulator, call the `stop()` method on the `LocalGcdHelper` object returned by `LocalGcdHelper.start()`.
33+
4. Stop the local datastore emulator by calling the `stop()` method, like so:
34+
```java
35+
helper.stop();
36+
```
3337

3438
#### On a remote machine
3539

36-
You can test against a remote datastore emulator as well. To do this, set the `DatastoreOptions` project endpoint to the hostname of the remote machine, like the example below.
40+
You can test against a remote Datastore emulator as well. To do this, set the `DatastoreOptions` project endpoint to the hostname of the remote machine, like the example below.
3741

3842
```java
3943
DatastoreOptions options = DatastoreOptions.builder()
40-
.projectId(PROJECT_ID)
44+
.projectId("my-project-id") // must match project ID specified on remote machine
4145
.host("http://<hostname of machine>:<port>")
46+
.authCredentials(AuthCredentials.noAuth())
4247
.build();
4348
Datastore localDatastore = options.service();
4449
```
4550

46-
Note that the remote datastore must be running before your tests are run.
51+
We recommend that you start the emulator on the remote machine using the [Google Cloud SDK](https://cloud.google.com/sdk/gcloud/reference/beta/emulators/datastore/) from command line, as shown below:
52+
53+
```
54+
gcloud beta emulators datastore start --host-port <hostname of machine>:<port>
55+
```
4756

4857
### Testing code that uses Storage
4958

50-
Currently, there isn't an emulator for Google Cloud Storage, so an alternative is to create a test project. `RemoteGcsHelper` contains convenience methods to make setting up and cleaning up the test project easier. To use this class, follow the steps below:
59+
Currently, there isn't an emulator for Google Cloud Storage, so an alternative is to create a test project. `RemoteStorageHelper` contains convenience methods to make setting up and cleaning up the test project easier. To use this class, follow the steps below:
5160

5261
1. Create a test Google Cloud project.
5362

5463
2. Download a JSON service account credentials file from the Google Developer's Console. See more about this on the [Google Cloud Platform Storage Authentication page][cloud-platform-storage-authentication].
5564

56-
3. Create a `RemoteGcsHelper` object using your project ID and JSON key.
57-
Here is an example that uses the `RemoteGcsHelper` to create a bucket.
65+
3. Create a `RemoteStorageHelper` object using your project ID and JSON key.
66+
Here is an example that uses the `RemoteStorageHelper` to create a bucket.
5867
```java
59-
RemoteGcsHelper gcsHelper =
60-
RemoteGcsHelper.create(PROJECT_ID, new FileInputStream("/path/to/my/JSON/key.json"));
61-
Storage storage = gcsHelper.options().service();
62-
String bucket = RemoteGcsHelper.generateBucketName();
68+
RemoteStorageHelper helper =
69+
RemoteStorageHelper.create(PROJECT_ID, new FileInputStream("/path/to/my/JSON/key.json"));
70+
Storage storage = helper.options().service();
71+
String bucket = RemoteStorageHelper.generateBucketName();
6372
storage.create(BucketInfo.of(bucket));
6473
```
6574

@@ -68,7 +77,7 @@ Here is an example that uses the `RemoteGcsHelper` to create a bucket.
6877
5. Clean up the test project by using `forceDelete` to clear any buckets used.
6978
Here is an example that clears the bucket created in Step 3 with a timeout of 5 seconds.
7079
```java
71-
RemoteGcsHelper.forceDelete(storage, bucket, 5, TimeUnit.SECONDS);
80+
RemoteStorageHelper.forceDelete(storage, bucket, 5, TimeUnit.SECONDS);
7281
```
7382

7483
### Testing code that uses Resource Manager
@@ -134,4 +143,4 @@ Here is an example that clears the dataset created in Step 3.
134143
```
135144

136145
[cloud-platform-storage-authentication]:https://cloud.google.com/storage/docs/authentication?hl=en#service_accounts
137-
[create-service-account]:https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount
146+
[create-service-account]:https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount

gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/it/ITBigQueryTest.java

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@
6767
import com.google.gcloud.storage.BlobInfo;
6868
import com.google.gcloud.storage.BucketInfo;
6969
import com.google.gcloud.storage.Storage;
70-
import com.google.gcloud.storage.testing.RemoteGcsHelper;
70+
import com.google.gcloud.storage.testing.RemoteStorageHelper;
7171

7272
import org.junit.AfterClass;
7373
import org.junit.BeforeClass;
@@ -135,7 +135,7 @@ public class ITBigQueryTest {
135135
private static final String LOAD_FILE = "load.csv";
136136
private static final String JSON_LOAD_FILE = "load.json";
137137
private static final String EXTRACT_FILE = "extract.csv";
138-
private static final String BUCKET = RemoteGcsHelper.generateBucketName();
138+
private static final String BUCKET = RemoteStorageHelper.generateBucketName();
139139
private static final TableId TABLE_ID = TableId.of(DATASET, "testing_table");
140140
private static final String CSV_CONTENT = "StringValue1\nStringValue2\n";
141141
private static final String JSON_CONTENT = "{"
@@ -172,9 +172,9 @@ public class ITBigQueryTest {
172172
@BeforeClass
173173
public static void beforeClass() throws InterruptedException {
174174
RemoteBigQueryHelper bigqueryHelper = RemoteBigQueryHelper.create();
175-
RemoteGcsHelper gcsHelper = RemoteGcsHelper.create();
175+
RemoteStorageHelper storageHelper = RemoteStorageHelper.create();
176176
bigquery = bigqueryHelper.options().service();
177-
storage = gcsHelper.options().service();
177+
storage = storageHelper.options().service();
178178
storage.create(BucketInfo.of(BUCKET));
179179
storage.create(BlobInfo.builder(BUCKET, LOAD_FILE).contentType("text/plain").build(),
180180
CSV_CONTENT.getBytes(StandardCharsets.UTF_8));
@@ -200,7 +200,7 @@ public static void afterClass() throws ExecutionException, InterruptedException
200200
RemoteBigQueryHelper.forceDelete(bigquery, DATASET);
201201
}
202202
if (storage != null) {
203-
boolean wasDeleted = RemoteGcsHelper.forceDelete(storage, BUCKET, 10, TimeUnit.SECONDS);
203+
boolean wasDeleted = RemoteStorageHelper.forceDelete(storage, BUCKET, 10, TimeUnit.SECONDS);
204204
if (!wasDeleted && LOG.isLoggable(Level.WARNING)) {
205205
LOG.log(Level.WARNING, "Deletion of bucket {0} timed out, bucket is not empty", BUCKET);
206206
}

gcloud-java-core/src/main/java/com/google/gcloud/AuthCredentials.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -339,7 +339,7 @@ public RestorableState<AuthCredentials> capture() {
339339
/**
340340
* A placeholder for credentials to signify that requests sent to the server should not be
341341
* authenticated. This is typically useful when using the local service emulators, such as
342-
* {@code LocalGcdHelper} and {@code LocalResourceManagerHelper}.
342+
* {@code LocalDatastoreHelper} and {@code LocalResourceManagerHelper}.
343343
*/
344344
public static class NoAuthCredentials extends AuthCredentials {
345345

@@ -425,7 +425,7 @@ public static ServiceAccountAuthCredentials createFor(String account, PrivateKey
425425

426426
/**
427427
* Creates a placeholder denoting that no credentials should be used. This is typically useful
428-
* when using the local service emulators, such as {@code LocalGcdHelper} and
428+
* when using the local service emulators, such as {@code LocalDatastoreHelper} and
429429
* {@code LocalResourceManagerHelper}.
430430
*/
431431
public static AuthCredentials noAuth() {

0 commit comments

Comments
 (0)