Add a line break every fix number of byte
I am using AIX and ksh.
I have a very large file which does not have line break. I am sure that the file is generated from fixed record length file.
For example, if the file size is 123456789000, the file should contain 123456789 records, each 1000 byte.
Because missing line break char, i.e. LF, it is very difficult to handle this file.
I have tried the dd command using bs=1000. But, seems that cannot add a n after every block.
Using split command can break the file into 123456789 files, each 1000 byte size. This is quite a trouble then to continue.
Is there any way to add a n every 1000 byte ?
If I can add back n every 1000 byte, there will be more easy for me to handle the file.
BTW, prefer to use ksh script, no python, no perl. no C compiler.
Thanks in advance.
newline ksh line-breaks
asked Nov 23 '18 at 14:04
Alvin SIUAlvin SIU
708617
add a comment |
I am using AIX and ksh.
I have a very large file which does not have line break. I am sure that the file is generated from fixed record length file.
For example, if the file size is 123456789000, the file should contain 123456789 records, each 1000 byte.
Because missing line break char, i.e. LF, it is very difficult to handle this file.
I have tried the dd command using bs=1000. But, seems that cannot add a n after every block.
Using split command can break the file into 123456789 files, each 1000 byte size. This is quite a trouble then to continue.
Is there any way to add a n every 1000 byte ?
If I can add back n every 1000 byte, there will be more easy for me to handle the file.
BTW, prefer to use ksh script, no python, no perl. no C compiler.
Thanks in advance.
newline ksh line-breaks
asked Nov 23 '18 at 14:04
Alvin SIUAlvin SIU
708617
I didn;t test for large files or with AIX/ksh: Can you usefold -w 1000?
– Walter A
Nov 29 '18 at 12:31
@WalterA It works. I and my colleague really appreciate. This helps a lot. Please put it as an answer and I will choose it as accepted answer. Once again, thanks a lot.
– Alvin SIU
Dec 7 '18 at 13:16
add a comment |
I am using AIX and ksh.
I have a very large file which does not have line break. I am sure that the file is generated from fixed record length file.
For example, if the file size is 123456789000, the file should contain 123456789 records, each 1000 byte.
Because missing line break char, i.e. LF, it is very difficult to handle this file.
I have tried the dd command using bs=1000. But, seems that cannot add a n after every block.
Using split command can break the file into 123456789 files, each 1000 byte size. This is quite a trouble then to continue.
Is there any way to add a n every 1000 byte ?
If I can add back n every 1000 byte, there will be more easy for me to handle the file.
BTW, prefer to use ksh script, no python, no perl. no C compiler.
Thanks in advance.
newline ksh line-breaks
asked Nov 23 '18 at 14:04
Alvin SIUAlvin SIU
708617
I am using AIX and ksh.
I have a very large file which does not have line break. I am sure that the file is generated from fixed record length file.
For example, if the file size is 123456789000, the file should contain 123456789 records, each 1000 byte.
Because missing line break char, i.e. LF, it is very difficult to handle this file.
I have tried the dd command using bs=1000. But, seems that cannot add a n after every block.
Using split command can break the file into 123456789 files, each 1000 byte size. This is quite a trouble then to continue.
Is there any way to add a n every 1000 byte ?
If I can add back n every 1000 byte, there will be more easy for me to handle the file.
BTW, prefer to use ksh script, no python, no perl. no C compiler.
Thanks in advance.
newline ksh line-breaks
newline ksh line-breaks
asked Nov 23 '18 at 14:04
Alvin SIUAlvin SIU
708617
asked Nov 23 '18 at 14:04
Alvin SIUAlvin SIU
708617
asked Nov 23 '18 at 14:04
Alvin SIUAlvin SIU
708617
asked Nov 23 '18 at 14:04
Alvin SIUAlvin SIU
708617
asked Nov 23 '18 at 14:04
Alvin SIUAlvin SIU
708617
708617
I didn;t test for large files or with AIX/ksh: Can you usefold -w 1000?
– Walter A
Nov 29 '18 at 12:31
@WalterA It works. I and my colleague really appreciate. This helps a lot. Please put it as an answer and I will choose it as accepted answer. Once again, thanks a lot.
– Alvin SIU
Dec 7 '18 at 13:16
add a comment |
I didn;t test for large files or with AIX/ksh: Can you usefold -w 1000?
– Walter A
Nov 29 '18 at 12:31
@WalterA It works. I and my colleague really appreciate. This helps a lot. Please put it as an answer and I will choose it as accepted answer. Once again, thanks a lot.
– Alvin SIU
Dec 7 '18 at 13:16
I didn;t test for large files or with AIX/ksh: Can you use
fold -w 1000 ?– Walter A
Nov 29 '18 at 12:31
I didn;t test for large files or with AIX/ksh: Can you use
fold -w 1000 ?– Walter A
Nov 29 '18 at 12:31
@WalterA It works. I and my colleague really appreciate. This helps a lot. Please put it as an answer and I will choose it as accepted answer. Once again, thanks a lot.
– Alvin SIU
Dec 7 '18 at 13:16
@WalterA It works. I and my colleague really appreciate. This helps a lot. Please put it as an answer and I will choose it as accepted answer. Once again, thanks a lot.
– Alvin SIU
Dec 7 '18 at 13:16
add a comment |
3 Answers
3
active
oldest
votes
You can use
fold -w 1000 inputfile > outputfile
answered Dec 7 '18 at 13:55
Walter AWalter A
10.8k21031
add a comment |
That's a big line. You could try this:
while IFS= read -rn 1000 chars || [[ -n $chars ]]; do
printf "%sn" "$chars"
done < file.in > file.out
I wouldn't be surprised if you get errors.
answered Nov 23 '18 at 16:46
glenn jackmanglenn jackman
167k26147238
add a comment |
Not really elegant, but works on AIX:
#!/bin/sh
while dd bs=1000 count=1 2>ddstat.tmp; head -n1 ddstat.tmp |
grep -v -q '^0+0 records in'; do
printf 'n'
done <split.in >split.out
Edit: a version without temporary 'ddstat.tmp' file:
#!/bin/sh
exec 4>split.out
while dd bs=1000 count=1 2>&1 >&4 | head -n1 |
grep -v -q '^0+0 records in'; do
printf 'n' >&4
done <split.in
answered Nov 23 '18 at 17:30
Lorinczy ZsigmondLorinczy Zsigmond
9101715
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53448139%2fadd-a-line-break-every-fix-number-of-byte%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can use
fold -w 1000 inputfile > outputfile
answered Dec 7 '18 at 13:55
Walter AWalter A
10.8k21031
add a comment |
You can use
fold -w 1000 inputfile > outputfile
answered Dec 7 '18 at 13:55
Walter AWalter A
10.8k21031
add a comment |
You can use
fold -w 1000 inputfile > outputfile
answered Dec 7 '18 at 13:55
Walter AWalter A
10.8k21031
You can use
fold -w 1000 inputfile > outputfile
answered Dec 7 '18 at 13:55
Walter AWalter A
10.8k21031
answered Dec 7 '18 at 13:55
Walter AWalter A
10.8k21031
answered Dec 7 '18 at 13:55
Walter AWalter A
10.8k21031
answered Dec 7 '18 at 13:55
Walter AWalter A
10.8k21031
10.8k21031
add a comment |
add a comment |
That's a big line. You could try this:
while IFS= read -rn 1000 chars || [[ -n $chars ]]; do
printf "%sn" "$chars"
done < file.in > file.out
I wouldn't be surprised if you get errors.
answered Nov 23 '18 at 16:46
glenn jackmanglenn jackman
167k26147238
add a comment |
That's a big line. You could try this:
while IFS= read -rn 1000 chars || [[ -n $chars ]]; do
printf "%sn" "$chars"
done < file.in > file.out
I wouldn't be surprised if you get errors.
answered Nov 23 '18 at 16:46
glenn jackmanglenn jackman
167k26147238
add a comment |
That's a big line. You could try this:
while IFS= read -rn 1000 chars || [[ -n $chars ]]; do
printf "%sn" "$chars"
done < file.in > file.out
I wouldn't be surprised if you get errors.
answered Nov 23 '18 at 16:46
glenn jackmanglenn jackman
167k26147238
That's a big line. You could try this:
while IFS= read -rn 1000 chars || [[ -n $chars ]]; do
printf "%sn" "$chars"
done < file.in > file.out
I wouldn't be surprised if you get errors.
answered Nov 23 '18 at 16:46
glenn jackmanglenn jackman
167k26147238
answered Nov 23 '18 at 16:46
glenn jackmanglenn jackman
167k26147238
answered Nov 23 '18 at 16:46
glenn jackmanglenn jackman
167k26147238
answered Nov 23 '18 at 16:46
glenn jackmanglenn jackman
167k26147238
167k26147238
add a comment |
add a comment |
Not really elegant, but works on AIX:
#!/bin/sh
while dd bs=1000 count=1 2>ddstat.tmp; head -n1 ddstat.tmp |
grep -v -q '^0+0 records in'; do
printf 'n'
done <split.in >split.out
Edit: a version without temporary 'ddstat.tmp' file:
#!/bin/sh
exec 4>split.out
while dd bs=1000 count=1 2>&1 >&4 | head -n1 |
grep -v -q '^0+0 records in'; do
printf 'n' >&4
done <split.in
answered Nov 23 '18 at 17:30
Lorinczy ZsigmondLorinczy Zsigmond
9101715
add a comment |
Not really elegant, but works on AIX:
#!/bin/sh
while dd bs=1000 count=1 2>ddstat.tmp; head -n1 ddstat.tmp |
grep -v -q '^0+0 records in'; do
printf 'n'
done <split.in >split.out
Edit: a version without temporary 'ddstat.tmp' file:
#!/bin/sh
exec 4>split.out
while dd bs=1000 count=1 2>&1 >&4 | head -n1 |
grep -v -q '^0+0 records in'; do
printf 'n' >&4
done <split.in
answered Nov 23 '18 at 17:30
Lorinczy ZsigmondLorinczy Zsigmond
9101715
add a comment |
Not really elegant, but works on AIX:
#!/bin/sh
while dd bs=1000 count=1 2>ddstat.tmp; head -n1 ddstat.tmp |
grep -v -q '^0+0 records in'; do
printf 'n'
done <split.in >split.out
Edit: a version without temporary 'ddstat.tmp' file:
#!/bin/sh
exec 4>split.out
while dd bs=1000 count=1 2>&1 >&4 | head -n1 |
grep -v -q '^0+0 records in'; do
printf 'n' >&4
done <split.in
answered Nov 23 '18 at 17:30
Lorinczy ZsigmondLorinczy Zsigmond
9101715
Not really elegant, but works on AIX:
#!/bin/sh
while dd bs=1000 count=1 2>ddstat.tmp; head -n1 ddstat.tmp |
grep -v -q '^0+0 records in'; do
printf 'n'
done <split.in >split.out
Edit: a version without temporary 'ddstat.tmp' file:
#!/bin/sh
exec 4>split.out
while dd bs=1000 count=1 2>&1 >&4 | head -n1 |
grep -v -q '^0+0 records in'; do
printf 'n' >&4
done <split.in
answered Nov 23 '18 at 17:30
Lorinczy ZsigmondLorinczy Zsigmond
9101715
edited Nov 25 '18 at 5:33
answered Nov 23 '18 at 17:30
Lorinczy ZsigmondLorinczy Zsigmond
9101715
answered Nov 23 '18 at 17:30
Lorinczy ZsigmondLorinczy Zsigmond
9101715
answered Nov 23 '18 at 17:30
Lorinczy ZsigmondLorinczy Zsigmond
9101715
9101715
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53448139%2fadd-a-line-break-every-fix-number-of-byte%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I didn;t test for large files or with AIX/ksh: Can you use
fold -w 1000?– Walter A
Nov 29 '18 at 12:31
@WalterA It works. I and my colleague really appreciate. This helps a lot. Please put it as an answer and I will choose it as accepted answer. Once again, thanks a lot.
– Alvin SIU
Dec 7 '18 at 13:16