-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy patha8rawconv-manual.html
More file actions
1746 lines (1652 loc) · 90.2 KB
/
a8rawconv-manual.html
File metadata and controls
1746 lines (1652 loc) · 90.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<html>
<head>
<meta http-equiv="content-type" content="text-html; charset=utf-8"></meta>
<title>a8rawconv</title>
<style>
body {
font-family: Verdana, sans-serif;
font-weight: normal;
font-size: 100%;
line-height: 1.3;
background: white;
color: black;
margin: 0;
}
h1 {
font-size: 20pt;
}
h2 {
font-size: 16pt;
font-weight: bold;
border-bottom: 1px solid black;
margin-top: 2.5em;
}
h3 {
font-weight: bold;
font-size: 12pt;
margin-top: 2.5em;
}
tt {
font-family: monospace;
color: #600;
}
b {
font-weight: bold;
}
pre {
margin: 0.5em 2em;
line-height: 1;
font-family: monospace;
color: #600;
}
dl {
margin: 0 0 0 2em;
}
dt {
font-style: italic;
}
dd {
margin: 0.5em 0 0.5em 2em;
}
dl.version dd {
margin-left: 0;
}
li {
margin: 0.5em 0 0.5em 0;
}
.dense li {
margin: 0;
}
pre span {
font-family: monospace;
line-height: 1;
}
.hot {
color: red;
}
.tocframe {
position: fixed;
left: 0px;
top: 0px;
width: 20%;
height: 100%;
overflow: auto;
}
.toc {
margin: 0.5em;
}
.toc p {
font-size: 80%;
margin-top: 0.25em;
margin-bottom: 0.25em;
line-height: 1.1;
}
p.toc1 {
margin-top: 1.0em;
margin-bottom: 0.25em;
padding-left: 0.5em;
text-indent: -0.5em;
font-size: 100%;
padding-bottom: 2px;
}
p.toc2 {
padding-left: 1.5em;
text-indent: -0.5em;
}
p.toc3 {
padding-left: 2.5em;
text-indent: -0.5em;
}
.mainframe {
position: fixed;
left: 21%;
margin: 0;
height: 100%;
overflow: auto;
}
a {
text-decoration: none;
}
a:link {
color: #00f;
}
a:visited {
color: #80a;
}
.toc a:visited {
color: #00f;
}
@media (prefers-color-scheme: dark) {
body {
background: #222;
color: #ddd;
}
h2 {
border-color: #eee;
}
pre {
color: #c94;
}
tt {
color: #c94;
}
a:link {
color: #4af;
}
a:visited {
color: #96f;
}
.toc a:visited {
color: #4af;
}
}
</style>
</head>
<body>
<div id="mainframe" class="mainframe">
<h1>a8rawconv</h1>
<p>
Copyright © 2014-2023 Avery Lee
</p>
<h2>Introduction</h2>
<p>
<i>a8rawconv</i> is a utility for converting between raw and decoded disk images,
taking advantage of recent availability of high-quality devices for reading and
writing raw flux-level images of floppy disks. This allows for easy and fast
methods of creating usable images of floppy disks without a modified disk drive,
particularly unusually encoded or even copy protected floppy disks. Decoded disk
images are better suited for testing in an emulator or on real hardware using
a disk simulator, and provide a way to verify whether a disk image is valid before
archiving.
</p>
<h2>Basic usage</h2>
<h3>Running the program (Windows)</h3>
<p>
The converter is provided in binary form as a Win32 command line utility called
<tt>a8rawconv.exe</tt>, which can be run on Windows 7 or later, on either
a 32-bit or 64-bit system. To use it, open a Command Prompt and run the program
from the command line. This allows command-line arguments to be specified and
the output to be seen.
</p>
<p>
Source code is also provided so that the converter can be rebuilt or modified.
Visual Studio 2022 or newer is required.
</p>
<h3>Running the program (Linux or macOS)</h3>
<p>
Although binaries are not provided for other platforms, the source code is included
so that it can be compiled. The core is designed to be portable to most systems that have at
least a C++11 compiler, with the serial device functionality also available
for Linux systems.
</p>
<p>
To compile the converter, build <tt>compileall.cpp</tt> with a C++ compiler
in C++11 or C++14 mode. This file is a bulk file that includes all the other
files, avoiding the need for a build system. This is the same build process
used for the Release build on Windows. Typically, this will suffice (replace
with your preferred C++ build system as appropriate):
</p>
<blockquote>
<tt>g++ -O2 -std=c++14 compileall.cpp</tt>
</blockquote>
<h3>Command-line syntax</h3>
<p>
<i>a8rawconv</i> is a command-line program that is invoked with the following
syntax:
</p>
<blockquote>
<tt>a8rawconv <i>[options]</i> <i>source-image</i> <i>destination-image</i></tt>
</blockquote>
<p>
Basic operations just take two filenames, the path to the source file and the
path to the destination file. The specific operation, such as decoding, encoding,
or format conversion, is determined by the filename extensions.
</p>
<p>
Many options are also supported to tweak the operation of the program. The list
of options is documented within the program itself, which can be seen simply
by running the program with no arguments:
</p>
<blockquote>
<tt>a8rawconv</tt>
</blockquote>
<h3>Decoding raw disk images</h3>
<p>
The main use of <i>a8rawconv</i> is to convert from a raw disk image produced
by imaging hardware to a more usable decoded disk image. This can be done by
running <i>a8rawconv</i> with the input and output names:
</p>
<blockquote>
<tt>a8rawconv track00.0.raw mydisk.atx</tt><br/>
<tt>a8rawconv mydisk.scp mydisk.atx</tt>
</blockquote>
<p>
Raw images can be produced from physical floppy disks either by KryoFlux or
SuperCard Pro imaging hardware. By default, the input format, decoding parameters and output format
are determined by the extension on the input and output filenames; <tt>.atx</tt> selects
the VAPI/ATX format with Atari 8-bit decoding parameters. Similarly, <tt>.nib</tt>
selects the NIB disk format with Apple II 5.25" decoding parameters. These
can be overridden with the <tt>-if</tt>, <tt>-of</tt>, and <tt>-d</tt> parameters.
</p>
<p>
When using KryoFlux raw streams (track00.0.raw, track02.0.raw, etc.), a 96/135 tracks per inch (TPI)
drive and an 70/80 track dump are assumed by default. Thus, the converter will dump
even tracks. When using a 48 TPI drive and 35/40 track dumps, the <tt>-tpi</tt>
switch must be used to change the interpretation to 48 TPI.
</p>
<p>
Side 2 of a KryoFlux double-sided raw stream set can be decoded as a single-sided disk
by specifying <tt>track00.1.raw</tt> as the starting filename. (This will only work if the
disk drive has been adjusted, since normally the top and bottom heads are offset.)
</p>
<h3>Interoperability problem with SuperCard Pro images</h3>
<p>
The SuperCard Pro (.scp) format has some issues with ambiguity in the way that tracks
are stored in the file, which can make it difficult to reliably determine whether
the tracks are stored with physical 48tpi or 96tpi spacing. This can cause interoperability
problems between programs where one program cannot read <tt>.scp</tt> images written
by another program. Current versions of <i>a8rawconv</i> use heuristics to try to
interpret the file, but incompatibilities may still occur, particularly depending
on the value of the disk type ID in the SCP image header.
</p>
<p>
To combat this problem, the <tt>-if</tt> and <tt>-of</tt> switches can be used to force
a particular interpretation of the track list. All four combinations of single-sided,
double-sided, 40-track (48tpi), and 80-track (96tpi) are supported. For instance, if
the other program is expecting the track list to be emitted with two sides and 40-track
spacing, the <tt>scp-ds40</tt> format ID will cause <i>a8rawconv</i> to read and write
the <tt>.scp</tt> file with this layout.
</p>
<p>
For Atari disks specifically, <i>a8rawconv</i> follows the actual behavior of the official
SuperCard Pro software up through version 2.20, which is to write images with
the Atari 400/800 disk type ID with a track layout of 40 tracks of two sides each in
the first 80 entries of the track list. This
means that for a single sided image, only every other track entry is populated or used.
This is regardless of whether the image header specifies 48tpi or 96tpi, as that flag
indicates the properties of the drive used and not the image layout.
</p>
<h2>Imaging physical floppy disks</h2>
<p>
Typically, the first step is to create a raw image of a floppy disk using the software
that comes with the imaging hardware. There are a few things that should be kept in
mind when doing this:
</p>
<ul>
<li>
Use the preservation modes of the imaging software when possible. This
typically records five revolutions of all tracks on the disk. For older
disk formats this often over-images the disk, but that's much better than
finding out later you're missing part of it. Another reason to do this
is that if the physical disk is deteriorating, you only want to do one
pass on the disk because you may not get another chance. The safest approach
is simply to grab a full five revolutions of all 80 tracks on both sides
and sort it out later.
</li>
<li>
For Atari 8-bit disks, you <i>must</i> have at least two revolutions imaged
per track, and you must image 40 tracks. An index-aligned, one-rev image
will not work because sectors can cross the index mark. A single revolution
in splice mode also will not work.
</li>
<li>
Atari 8-bit disks have 40 tracks. Recording with a straight C64 or Apple II
profile won't work, because you'll only have 35 out of the 40 tracks.
</li>
<li>
More revolutions are better because <i>a8rawconv</i> can use the duplicate
data to salvage marginal sectors or distinguish stable from weak sectors.
</li>
<li>
Currently, <i>a8rawconv</i> expects 80 track images when reading KryoFlux raw
streams by default. 40-track (48 TPI) image sets can be used by supplying the
<tt>-tpi 48</tt> switch when decoding.
</li>
<li>
While Atari drives spin at 288 RPM, and the PC drives used for imaging
spin at either 300 RPM or 360 RPM, either speed is fine for imaging. The
converter detects the speed of the imaging drive from the raw image and
adjusts the decoding parameters for the difference in speed.
</li>
</ul>
<p>
<i>a8rawconv</i> supports both KryoFlux raw streams (*.raw) and SuperCard Pro images
(*.scp). Although there are differences between both the KryoFlux and SuperCard Pro
hardware and software, both produce images suitable for conversion and neither has
a significant advantage over the other in image quality or read reliability.
</p>
<p>
When archiving, it is always recommended that you keep the original raw images of
a disk for archival purposes even if the converter successfully creates a fully
functional decoded disk. This is because the decoded disk does not contain all of
the information of the original disk. If it turns out that something is missing or
the converter is updated, the raw source can be used to reconvert without using
the physical disk again. Raw images compress well using standard compression tools.
</p>
<p>
In non-archival cases, this can be skipped. If you are decoding a disk
containing a BASIC program you wrote as a kid, there won't be any copy protection
on the disk and it's unlikely you'll care about the exact sector timing.
Therefore, once you verify that the decoded disk is good, you don't need to
keep the raw image. The raw images are so small in modern terms, though, that you
might consider still keeping them around.
</p>
<p>
<b>Note:</b> Some versions of the SuperCard Pro software have a buggy profile for
Atari 8-bit disks and will attempt to create an image using a single index-aligned
revolution. This will result in lost sectors on most disks due to sectors being
split across the index mark and <i>a8rawconv</i>
will issue a warning if it sees this in the image. The imaging parameters must be
manually adjusted to at least two revolutions.
</p>
<h3>Reading directly from physical floppy disks</h3>
<p>
<i>a8rawconv</i> can also use a SuperCard Pro device to directly read from
a floppy disk. The SuperCard Pro software must be installed first to install
the USB communications driver, after which the converter can access the hardware
using special filenames:
</p>
<ul>
<li>(Windows):</li>
<ul>
<li><tt>scp0:96tpi</tt> (read from first drive, using 96tpi track density)</li>
<li><tt>scp1:48tpi</tt> (read from second drive, using 48tpi track density)</li>
<li><tt>scp0:96tpi:com4</tt> (read from first drive as 96tpi, overriding device path to COM4)</li>
</ul>
<li>(Linux):</li>
<ul>
<li><tt>scp0:96tpi:/dev/ttyUSB0</tt> (read from first drive, using 96 tpi track density, from device <tt>/dev/ttyUSB0</tt>)</li>
</ul>
</ul>
<p>
Therefore, a physical disk can be imaged as follows:
</p>
<blockquote>
<tt>a8rawconv scp0:96tpi test.scp</tt>
</blockquote>
<p>
On Windows, <i>a8rawconv</i> will automatically search for the first COM port that is
a USB connection to a SuperCard Pro device, specifically an FTDI-based USB serial port
driver identified as VID_0403+PID_6015+SCP-JIMA. The Virtual COM Port (VCP) mode of the FTDI
serial driver must be enabled for this to work. It can be enabled in Device Manager.
</p>
<p>
On Linux, the serial device name must be
manually specified, and you may also need to run with elevated (superuser) privileges.
</p>
<p>
Reading directly from a floppy disk can be useful when doing testing or transferring
data. It is not recommended for old floppy disks since a raw image is not created during
the process, and repeating the decoding process with different settings requires re-reading
the disk, which is slower and puts more stress on the disk. Archiving should always be done
by creating a preservation-level raw image once and then re-decoding the raw image as
necessary.
</p>
<h3>Imaging to flux</h3>
<p>
When imaging from a physical disk, the destination can either be a raw flux image
(e.g. <tt>disk.scp</tt>) or a decoded image type (e.g. <tt>disk.atx</tt>). The best
target depends on the disk. In most cases, it is best to image directly to a raw flux
image first:
</p>
<blockquote>
<tt>a8rawconv scp0:96tpi disk.scp</tt>
</blockquote>
<p>
This will by default record a full five revolutions of each track in the image, in one
pass on the disk. The image can be then be decoded multiple times from that image
without requiring another pass on the original disk. More importantly, it can be
<i>re-decoded</i> later again if something is discovered problematic with an earlier
decode. Taking a preservation-level image of the disk in this manner is the safest
way to deal with rare disks that may be starting to fail.
</p>
<p>
There are a couple of other switches that can be useful when doing a raw image. One
is the <tt>-revs</tt> switch, which sets the revolution count when reading from an
SCP device. The range is 1-5 revs, though 2 revs are minimum for non-index aligned
disks like Atari disks. Reducing the rev count with <tt>-revs 2</tt> is useful for
faster imaging of a non-critical disk in good condition, and will also produce a
smaller <tt>.scp</tt> image.
</p>
<p>
The other useful switch is <tt>-g</tt>, which overrides the disk geometry. a8rawconv
will normally determine the best geometry based on the input and output formats, but
for a raw imaging operation there is no such information. In this case, it will
assume single-sided 40 tracks, which is enough to image up to Atari double-density
but may be inadequate for other formats. <tt>-g</tt> allows the track and side
count to be overridden, such as <tt>-g 80,2</tt> for double-sided 80 tracks. This
also allows overdumping up to 42 or 84 tracks for disks that have extra information
outside of the normal track range. For truly blind cases where the disk format is
totally unknown, it can be a good idea to just dump a full 83 tracks on both sides --
the unused tracks can just be ignored later if they turn out to not have any data.
</p>
<h3>Imaging to a decoded image</h3>
<p>
On the other hand, sometimes a disk isn't that precious and the two step process isn't
convenient. For instance, if you just wrote out a disk from a 1050 and just want to
image it, you can go directly from the physical disk to decoded image:
</p>
<blockquote>
<tt>a8rawconv scp0:96tpi disk.atx</tt>
</blockquote>
<p>
In this case, <i>a8rawconv</i> will both read and decode the disk in one pass. Of course,
if something is wrong and the decoding parameters need to be adjusted, the disk will
also need to be read again.
</p>
<h3>Assessing the quality of an image</h3>
<p>
The converter has a few features to assess the quality of a decoded image. The
simplest is just how many sectors were extracted. For Atari 8-bit single density
disks — the most common format — the converter should report no missing
sectors or sectors with errors:
</p>
<pre>Writing ATX file: e:\test.atx
0 missing sectors, 0 phantom sectors, 0 sectors with errors</pre>
<p>
On the other hand, you might see something like this:
</p>
<pre>Writing ATX file: e:\test.atx
WARNING: Track 20, sector 16: Multiple sectors found at the same position
0.81 but different bad data. Encoding weak sector at offset 25.
WARNING: Track 20: Missing sectors: 18.
1 missing sector, 0 phantom sectors, 1 sector with errors</pre>
<p>
If your disk is supposed to be just a normal DOS disk, this means you have some
problems. If it's a commercial release, and especially if it is copy protected,
then this might be expected. In the latter case, some experience with copy
protection techniques is helpful and testing the decoded disk image is recommended.
</p>
<h3>Tweaking the decoding period</h3>
<p>
<i>a8rawconv</i> is designed to have good defaults so that most of the time
conversion can be performed without a string of command-line switches. However,
sometimes tweaking is necessary. One useful switch is the <tt>-p</tt> switch,
which tweaks the default bit cell period used by the decoder, as a percentage of
the normal value. This can be used
to shrink or stretch the expected length of a bit slightly to nudge the decoder
toward success when it is having trouble with a marginal disk:
</p>
<blockquote>
<tt>a8rawconv -p 98 disk.scp disk.atx</tt>
</blockquote>
<p>
The recommended approach is to try small adjustments in the range of 95-105 and
watch the number of errors to zero in on the best value. Don't try crazy values
like 70 or 130 to start with; you're likely to just get complete failure to
decode the track. Watch out for when the decoder starts missing sectors entirely,
as past that point the error count may drop because the decoder stops seeing
some of the sectors.
</p>
<p>
The <tt>-t</tt> switch is also useful when tweaking decoding settings. It limits
decoding to a single track on the disk, filtering out errors from tracks other
than the one you're focusing on:
</p>
<blockquote>
<tt>a8rawconv -t 27 disk.scp disk.atx</tt>
</blockquote>
<p>
Note that this also causes only that track to be emitted into the output file, so
this should not be used to produce the final disk image file. When reading from
a SuperCard Pro, this will seek the mechanism to and read only the selected track.
</p>
<h3>Post-compensation</h3>
<p>
Floppy disk media is subject to a <i>peak shift</i> effect where flux transitions
that are close together can have their read signals interact such that the pulses read
farther apart than they were originally written. This effect can be serious enough to
push flux transitions out of their original bit cell timing. For 4us FM and GCR formats,
this is typically ignored, but for 2us MFM formats this is usually handled on the
write side with <i>write precompensation</i>, where patterns known to cause this
problem are pre-adjusted to combat the peak shift effect and read with approximately
intended timing.
</p>
<p>
A notable exception to this is Macintosh 800K disks, which can show severe peak shift
effects when read with a standard, constant-speed drive. This is partly due to the
high flux density, with the 2us GCR format producing minimum transition spacing
35% tighter than a 2us MFM format. To combat this, <i>a8rawconv</i> by default applies
<i>post-compensation</i> on the flux pattern when decoding to or analyzing for
the Macintosh GCR encoding format. This moves the transitions back closer to each
other so they are sufficiently within bitcell timing. Currently post-compensation
is not used for any other encoding as they are assumed to either not need it or to
have already had write precompensation applied.
</p>
<p>
If for some reason this default behavior is unsuitable, the post-compensation mode can
be overridden with the <tt>-P</tt> switch.
</p>
<h3>Weak sectors</h3>
<p>
When multiple revolutions are available for a track, <i>a8rawconv</i> compares
all copies of each physical sector to see whether they are the same. If they
are not, it attempts to separate them into good and bad copies, and reports
when this occurs:
</p>
<pre>WARNING: Track 38, sector 1: 1/3 bad sector reads discarded at position 0.25.</pre>
<p>
For a standard unprotected disk, this is a bad sign, as it means that the disk or either the
imaging hardware is marginal, and could mean that either the disk is going bad
or the imaging drive needs to be cleaned. Fortunately, the Cyclic Redundancy Check (CRC)
stored with each sector allows the converter to tell if a sector is good, and this
message means that the converter was able to recover the sector. This is one reason
that five disk revolutions is recommended — the extra images of each sector increase
the changes that the converter will be able to recover sectors from marginal disks.
</p>
<p>
When tweaking the decoder timing with <tt>-p</tt>, the ratio of good to bad sector
reads can be used to aid in determining whether a change in timing is helping or hurting.
</p>
<p>
Note that in this case, "good" means that the sector was read correctly. Protected
disks often have sectors that were written with deliberate CRC errors such that they
are always read by the disk drive with errors; the converter can isolate good reads
of these sectors in the same way, as both the CRC and data will be stable even if
the CRC is incorrect.
</p>
<p>
In the event that the converter can't find at least two reads of the same sector that
match, it will salvage as much common data as it can and then encode a <i>weak sector</i>:
</p>
<pre>WARNING: Track 38, sector 4: Multiple sectors found at the same position
0.65 but different bad data. Encoding weak sector at offset 44.</pre>
<p>
For a regular disk, this is a bad sign as it means that the converter was not able
to get a clean read of the sector. For protected disks, though, this can be valid
as some disks were deliberately created with weak sectors that changed every time
they were read by the disk drive. Typically there are only a small handful of these
at most. When weak sectors are found, it is recommended
that the decode be retried with different <tt>-p</tt> settings to try to determine
whether these are truly weak sectors or bad reads of a stable sector.
</p>
<h3>Viewing track/sector layout</h3>
<p>
The <tt>-l</tt> switch causes the converter to display a crude representation
of the sectors within each track. This is useful to identify unusual disk
layouts:
</p>
<pre> 0 (18) | 6 8 10 12 14 16 18 1 3 5 7 9 11 13 15 17 2 4
1 (18) | 8 10 12 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6
2 (18) | 8 10 12 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6
3 (18) | 8 10 12 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6
4 (18) | 10 12 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6 8
5 (18) | 10 12 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6 8
6 (18) | 12 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6 8 10
7 (18) | 12 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6 8 10
8 (18) | 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12
9 (18) | 14 16 18 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12
10 (18) | 16 18 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14
11 (18) | 16 18 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14
12 (18) | 18 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14 16</pre>
<p>
The left columns indicate the track number and number of physical sectors.
The remainder of the line shows all sectors in the track, where the relative
position of each sector number indicates the angular position of that sector.
Remember that floppy disk tracks are a continuous circle, so the start and
end of the line are the same position on the disk.
</p>
<p>
For a disk formatted on a standard Atari 810 or 1050 drive, even and odd
sectors will be separated due to <i>sector interleave</i>, a reordering
of the sectors to improve read/write performance. Disk drives with high-speed
capability can format with a different sector interleave for better high-speed
performance. Some commercial disks were written with a non-standard interleave
and read slower than usual on a real drive.
</p>
<p>
The sectors on successive tracks will also be offset due to skew between the tracks.
This is related to the delay when stepping between tracks during initial formatting.
It is also possible for tracks to lack such skew and be aligned to the index
mark:
</p>
<pre> 0 (18) | 18 7 14 3 10 17 6 13 2 9 16 5 12 1 8 15 4 11
1 (18) | 18 7 14 3 10 17 6 13 2 9 16 5 12 1 8 15 4 11
2 (18) | 18 7 14 3 10 17 6 13 2 9 16 5 12 1 8 15 4 11
3 (18) | 18 7 14 3 10 17 6 13 2 9 16 5 12 1 8 15 4 11
4 (18) | 18 7 14 3 10 17 6 13 2 9 16 5 12 1 8 15 4 11
5 (18) | 18 7 14 3 10 17 6 13 2 9 16 5 12 1 8 15 4 11
6 (18) | 18 7 14 3 10 17 6 13 2 9 16 5 12 1 8 15 4 11</pre>
<p>
Such a disk will read and write fine, but generally indicates
that the disk was not created in an Atari drive. This example also shows
a different sector interleave, which can cause the Atari to read the disk
more slowly than usual.
</p>
<p>
Highly unusual patterns typically indicate copy protection, such as this
track with 34 physical sectors:
</p>
<pre> 2 (34) | 2 34 56 7 8 9 10112 114 116 11 23 45 67 89 10111213115 117 1</pre>
<p>
Any track on a single-density Atari disk with more than 18 physical sectors
on it has more than one duplicated or <i>phantom</i> sectors, where two
or more physical sectors have the same sector number. This is used for
protection schemes where the timing of the sector read determines which
of the phantom sectors is read. Such a disk cannot be created on a stock
Atari drive, although it can be created with a modified drive or with
a hardware imager such as SuperCard Pro.
</p>
<p>
Finally, there is the possibility that the disk is just simply something else:
</p>
<pre> 0 ( 9) | 1 2 3 4 5 6 7 8 9
1 ( 9) | 1 2 3 4 5 6 7 8 9
2 ( 9) | 1 2 3 4 5 6 7 8 9
...
Writing ATX file: e:\test.atx
WARNING: Track 0, sector 1: Long sector of 512 bytes found.
WARNING: Track 0, sector 2: Long sector of 512 bytes found.
WARNING: Track 0, sector 3: Long sector of 512 bytes found.
</pre>
<p>
For Atari disk formats, the converter attempts both FM and MFM decoding on
each track, which means that it can pick up even non-Atari disk formats.
In this case, the disk has 9 index-aligned sectors per track with 512 bytes per track
instead of 18 sectors per track of 128 bytes each, meaning that it is
likely a 360K MS-DOS disk, not an Atari disk.
</p>
<h3>Flippy disks</h3>
<p>
Some software shipped on "flippy" disks that could be flipped upside down to
access another program variant or more data on the other side of the disk.
Imaging these disks is problematic for a couple of reasons:
</p>
<ul>
<li>
Physically flipping the disk moves the index hole of the disk to the other
side. The old floppy disk drives used with computers such as the Atari 800
and Apple II were often configured such that the index hole was ignored.
PC drives, however, can require the index hole to read the disk, and may
simply refuse to read a flipped disk.
</li>
<li>
Track 0 on side 1 (bottom) is at the equivalent position for track -4 on
side 2 (top). This means that the flip side cannot simply be imaged in a
double-sided drive, as the first four tracks will be inaccessible.
</li>
</ul>
<p>
It is possible to modify a floppy disk drive to circumvent one of these two problems.
If the drive is modified to realign the second head, reading the disk image will
produce track streams that are backwards. In that case,
the <tt>-r</tt> option should be passed to <i>a8rawconv</i> so that it reads the
track images backwards to match.
</p>
<h3>Multi-format disks</h3>
<p>
Ingenious fellows in the past found ways to create floppy disks that booted on
more than one computer, by creatively interleaving sectors from the different
formats on the same disk. <i>a8rawconv</i> does not directly support mixed-format
disks, but can aid in detecting such disks and extracting out platform-specific
versions. Unusual sector sizes or missing sectors/tracks in the layout map,
particularly on tracks 1 or 18, are a possible sign of such a disk. In that
case, conversion can be attempted using output formats for all platforms.
</p>
<p>
It is highly recommended that preservation-quality images of such multi-format
disks be kept, as they are uncommon and currently can't be converted to a single
decoded image.
</p>
<h3>Bit-level diagnosis</h3>
<p>
For experts dealing with troublesome disks, the <tt>-v</tt> option causes the
converter to report additional information about the decoding process. There are
four levels of verbosity:
</p>
<ul>
<li>
<tt>-v</tt>: Reports summary information about all physical sector images found in
all revolutions.
</li>
<li>
<tt>-vv</tt>: Reports more detailed information on each sector.
</li>
<li>
<tt>-vvv</tt>: Reports bit-level decoding output:
<pre>00 FF 00 FF 00 FF 00 FF 00 FF 00 FF 00 FF 00 FF | 15.64
00 FF 00 FF 00 FF 00 FF 00 FF 00 FF 00 FF 00 FF | 15.71
00 FF 00 FF 01 FF 03 FE 07 FC 0F F8 1F F1 3E E3 | 15.64
7D <span class="hot">C7 FB</span> 8F F6 1F ED 3F DB 7F B7 FF 6F FF DF FF | 15.79
BF <span class="hot">FF 7F</span> FF FF FF FF FF FF FF FF FF FF FF FF FF | 15.74
FF <span class="hot">FF FE</span> FF FD FF FB FF F7 FF EF FF DF FF BF FF | 15.63
7F <span class="hot">FF FF</span> FF FF FF FF FF FF FF FF FF FF FF FF FF | 15.65
FF <span class="hot">FF FE</span> FF FC FF F9 FF F2 FF E4 FF C8 FF 91 FF | 15.86</pre>
<p>
Listed on the left is the complete dump of the shift register states, and
on the right is the number of bit cell times measured for each group of
16 bits, composed of 8 clock bits and 8 data bits interleaved together.
</p>
<p>
Because the relationship between clock and data bits and the byte alignment
is not known until a sync byte is found, the bit-level dump simply shows all
possible offsets. The layout of 16 shifts per line ensures that once the
sync byte is found, successive bytes will be in a column. Above, highlighted
in red, is the Data Address Mark (DAM) with $C7 clock pattern and $FB data byte,
followed by the first four data bytes of the sector.
</p>
<p>
Note that while byte alignment will always be consistent within a single address
field or data field, there is no requirement for the address and data fields
of a sector to share common byte alignment and the two will often be
skewed by some bit offset from each other.
</p>
</li>
<li>
<tt>-vvvv</tt>: Reports flux-level decoding output.
</li>
</ul>
<p>
The higher verbosity levels can produce very large amounts of output, so restricting
decoding to a single track with <tt>-t</tt> is highly recommended.
</p>
<h3>Flux analysis mode</h3>
<p>
Invoking <i>a8rawconv</i> with the <tt>-analyze</tt> switch selects flux analysis mode
instead of regular conversion mode. In this mode, the flux timing of a raw disk is
analyzed to show the distribution of flux timing and to give insight into either
problems with the raw disk or if the wrong decoder is being used. <tt>-analyze</tt>
takes one argument, the encoding mode to model to select appropriate bit cell timing.
No output filename is needed with <tt>-analyze</tt>.
</p>
<p>
When <tt>-analyze</tt> is used, a histogram is produced for each track (<tt>-t</tt> is
recommended to show one track at a time):
</p>
<pre>Track 0.0:
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
| | |
| | |
| || |
| || |
| || |
| || | .
| || |. |||
|| || || |||.
|| || || | ||||
|| || || |. ||||
|| || || || ||||
|| || || ||||||| | |
||. || || .|||||||.|.| .. ||||| |
||| |||.|||| |||||||||||| |||.|||||.|.|
-------------------------------------------------------------------------------------------
. 2us . 4us . 6us .</pre>
<p>
On a disk in good condition and without anomalies like bad sectors, the
histogram peaks should be cleanly separated and centered around the expected
timings for the encoding. In this case, this track uses Macintosh GCR
encoding, which has a 2us bitcell and a maximum run of two consecutive 0 bits
between 1 bits, so the expected timings are 2/4/6 us. As can be seen in the
above, however, there is poor separation between 2us and 4us regions, which
will make it hard for the decoder to distinguish <tt>11</tt> and <tt>101</tt>
patterns.
</p>
<p>
The above histogram was generated with post-compensation disabled. With the
default <tt>mac800k</tt> post-compensation mode enabled, the result is
better:
</p><pre>Track 0.0:
|
|
|
|
|
| .
| |
| |
| |
| |
| ||
| ||
| ||
| ||
| || .
| || .|
|||| ||
|||| ||
| |||| .||
| |||| |||
| |||| |||
|||||| |||
||||||. |||.
||||||| |||||
||||||| .||||| |.
|||||||| |||||| ||.
||||||||| |||||| ||||
||||||||| ||||||| |||||
||||||||| |||||||| .|||||
||||||||||.| |||||||| ||||||||
-------------------------------------------------------------------------------------------
. 2us . 4us . 6us .</pre>
<p>
The analyzer will work with any flux pattern regardless of the specified
encoding, but the encoding is used to calibrate the time scale. For instance,
using <tt>pc-360k</tt> will scale the time axis and use labels for a 2us
bit cell at 300 RPM. The analysis encoding setting will also determine the
default post-compensation mode unless overridden with the <tt>-P</tt> switch.
</p>
<h3>False successes</h3>
<p>
It is possible for the converter to believe it has successfully read
a sector when it hasn't. The CRC16 used by the Atari disk format and other 177x compatible
disk formats makes this unlikely. Unfortunately, 5.25" Apple II disks use a weak 8-bit XOR
checksum algorithm that is more prone to false negatives and can fail to
detect errors. This has been seen in practice with old disks that had many
marginal sectors when read.
</p>
<p>
The only truly reliable way to verify a disk is to examine it manually, either by
attempting to boot the decoded image or checking the files. The latter case can be
annoying as the Apple II standard for text has the high bit set. The <tt>-I</tt> switch
will invert bit 7 on decoded data so that text files can be more easily read in
a hex editor. The resulting image will not work in an Apple II environment, however.
</p>
<h2>Converting between image formats</h2>
<h3>Downconverting to a simpler format</h3>
<p>
<i>a8rawconv</i> can also convert between decoded disk image formats. In this case,
flux-level processing is skipped and the decoded sectors are converted directly.
One use for this is to 'deprotect' an ATX file to ATR, when the disk image has no
protection and does not need full timing data:
</p>
<blockquote>
<tt>a8rawconv test.atx test.atr</tt>
</blockquote>
<p>
This will only work if the ATX disk image is compatible with the ATR format, with
sectors of uniform size and no unusual sector encodings. Timing information is
dropped, any missing sectors are padded, and special sector information like
CRC errors or weak bits are discarded. Warnings are displayed when this occurs
to indicate that the image may not work, especially for copy protected images.
However, this can also occur with unprotected, marginal disks that simply had
bad sectors, and this may be fine if those sectors are not needed.
</p>
<h3>Upconverting to a more complex format</h3>
<p>
It is also possible to go the other way, by converting from ATR to ATX:
</p>
<blockquote>
<tt>a8rawconv test.atr test.atx</tt>
</blockquote>
<p>
In this case, <i>a8rawconv</i> will synthesize timing data for the disk image.
This can be useful to enforce a specific sector interleave order, which can't
be encoded in ATR but can in ATX. By default, the ATX image has sectors laid
out with the sector interleave and track-to-track skew typical for an Atari
1050 disk drive. This can be overridden with the <tt>-i</tt> switch, as
described in <a href="#overriding-the-default-interleave">Overriding the
default interleave</a>.
</p>
<h3>Converting between flux formats</h3>
<p>
A KryoFlux (<tt>.raw</tt>) image can be converted to SuperCard Pro format (<tt>.scp</tt>):
</p>
<blockquote>
<tt>a8rawconv track00.0.raw image.scp</tt>
</blockquote>
<p>
For this conversion, partial revolutions at the beginning and end of each track
image are dropped, and the flux timing samples are converted from KryoFlux 40ns
precision to SuperCard Pro 25ns precision. No sector analysis is performed, however,
and the flux samples are converted without interpreting them, so this is agnostic
to disk format and will work with non-Atari formats.
</p>
<h3>Reincarnating an image</h3>
<p>
Finally, the most unusual but occasionally useful conversion mode is from a file
format to the same file format:
</p>
<blockquote>
<tt>a8rawconv test.atx test2.atx</tt>
</blockquote>
<p>
In this case, <i>a8rawconv</i> will rewrite the disk image with the same contents
but a new structure. This is useful if data is intact in the file but there is
some issue with the structure that is causing compability problems with other
software; if <i>a8rawconv</i> can read the file, it can re-write it with a clean
structure. This includes fixing issues from older versions of <i>a8rawconv</i>.
</p>
<h2>Writing images to physical disks</h2>
<h3>Encoding and writing a pre-decoded image</h3>
<p>
With a SuperCard Pro device, <i>a8rawconv</i> can also write images back to
physical disks. This is not typically useful for distribution as no one distributes
software on physical floppy disks anymore, but can still be useful for testing or
if floppy disk is the only way to get software onto a real computer.
</p>
<p>
To do so, simply specify the SuperCard Pro as the output device instead of
the source:
</p>
<blockquote>
<tt>a8rawconv test.atr scp0:96tpi</tt>
</blockquote>
<p>
The converter will then encode the disk image to physical format and write them
to the physical disk through the SCP. Pre-erasing or preformatting the disk
is not necessary. Both Atari FM (single) and MFM (enhanced) formats can
be encoded and written back to disk. Note that on an 80 track (96 TPI) drive,
only the even tracks are written.
</p>
<p>
<tt>135tpi</tt> can also be specified as a synonym for <tt>96tpi</tt>, as 3.5"
drives are also 80 tracks but have 135 TPI track density.
</p>