fwrite ignores dateTimeAs for datetime sequences

[alistaire47]

If a datetime is generated by seq, fwrite writes it as an epoch timestamp, ignoring the default dateTimeAs = "ISO", likely because of an ALTREP issue. Explicitly setting dateTimeAs to "ISO" makes no difference.

Reprex:

library(data.table)
Sys.setenv(TZ = 'UTC')
tmp <- tempfile()

# epoch instead of ISO
dt <- data.table(x = seq(as.POSIXct('1970-01-01'), by = '1 sec', length = 6))
fwrite(dt, tmp)
cat(readLines(tmp), sep = '\n')
#> x
#> 0
#> 1
#> 2
#> 3
#> 4
#> 5

# works correctly
dt[, x := x + 1]
fwrite(dt, tmp)
cat(readLines(tmp), sep = '\n')
#> x
#> 1970-01-01T00:00:01Z
#> 1970-01-01T00:00:02Z
#> 1970-01-01T00:00:03Z
#> 1970-01-01T00:00:04Z
#> 1970-01-01T00:00:05Z
#> 1970-01-01T00:00:06Z

Session info

> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.4

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.5.2   prettycode_1.0.2 parallel_3.5.2   withr_2.1.2
[5] crayon_1.3.4

[ben-schwen]

This might be related to the storage mode of POSIXct.

dt = data.table(x = seq(as.POSIXct('1970-01-01'), by = '1 sec', length = 6))
dt2 = data.table(x = seq(as.POSIXct('1970-01-01'), by = '1 sec', length = 6)+1)
dt3 = data.table(x=.POSIXct(1:6))
dt4 = data.table(x=.POSIXct(as.numeric(1:6)))
fwrite(dt)
#> x
#> 0
#> 1
#> 2
#> 3
#> 4
#> 5
typeof(dt$x)
#> [1] "integer"
fwrite(dt2)
#> x
#> 1970-01-01T00:00:01Z
#> 1970-01-01T00:00:02Z
#> 1970-01-01T00:00:03Z
#> 1970-01-01T00:00:04Z
#> 1970-01-01T00:00:05Z
#> 1970-01-01T00:00:06Z
typeof(dt2$x)
#> [1] "double"
fwrite(dt3)
#> x
#> 1
#> 2
#> 3
#> 4
#> 5
#> 6
typeof(dt3$x)
#> [1] "integer"
fwrite(dt4)
#> x
#> 1970-01-01T00:00:01Z
#> 1970-01-01T00:00:02Z
#> 1970-01-01T00:00:03Z
#> 1970-01-01T00:00:04Z
#> 1970-01-01T00:00:05Z
#> 1970-01-01T00:00:06Z
typeof(dt4$x)
#> [1] "double"

Be aware to also cater for cases where integer and double representation diverge e.g.

dt5=data.table(x=.POSIXct(c(1700000000)), y=.POSIXct(c(1700000000L)))
fwrite(dt5)

[aksh08022006]

Hi, I’d like to work on this issue.

From the examples above, it looks like fwrite() behaves differently depending on whether POSIXct is stored as integer vs double (e.g. sequences created by seq()), likely interacting with ALTREP and internal type checks.

My plan is to:

Reproduce the reported cases locally (including the integer/double divergence)

Trace how fwrite() detects and formats POSIXct columns internally

Check where dateTimeAs is bypassed for ALTREP or integer-backed datetime vectors

Propose a fix so dateTimeAs is respected consistently, regardless of storage mode

Before proceeding, I wanted to confirm:

Is the expected behavior that dateTimeAs should always be honored for POSIXct, independent of ALTREP or underlying storage?

Are there any known constraints or edge cases I should be careful about (e.g. performance or integer overflow)?

Happy to start with a focused PR once this approach is confirmed.

[ben-schwen]

Care to elaborate why you think that it is an ALTREP issue?

Besides that the approach looks fine.